Target-free Extrinsic Calibration of a 3D Lidar and an IMU

Overview

imu_lidar_calibration

Target-free Extrinsic Calibration of a 3D Lidar and an IMU

Overview

This repository is a toolkit for calibrating the 6-DoF rigid transformation between a 3D LIDAR and an IMU. It's based on an Extended Kalman Filter based algorithm which exploits the motion based calibration constraint for state update. This algorithm does not depend on any calibration target or special environmental features, like planes, for determining the extrinsic calibration between a 3D-Lidar and an IMU.

Presentation Video

Paper

Prerequisites

This code base was tested and implemented in a Ubuntu 16.04 system.

I realized that it may be tedious to build this on newer Ubuntu, hence docker. Although, I have provided the script to run this, basic knowledge of docker is necessary.

Install

  • Clone the source code for ndt_omp and build it in your catkin workspace
  • Clone this code-base and build it in your catkin workspace

Sensor suite

As far the 3D Lidar is considered, currently this code-base supports Ouster-128 but it is easy to expand for other 3D Lidars. We have tested this code-base by downsampling our Ouster 128 lidar to 64, 32, 16 channel modes. The important pre-requisite is that the points in lidar pointcloud must come with a measurement/firing timestamp. We use a Vectornav VN 300 IMU. In our setup the Lidar outputs scans at 10 Hz and the IMU outputs measurements at 400 Hz, however, we have also tested our algorithm with IMU running at 50 Hz, 100 Hz & 200 Hz.

alt text

Procedure

Data collection for extrinsic calibration

We need to excite all degrees of freedom during collecting data required for extrinsic calibration. An example video can be found here: Data collection procedure

Please find a sample dataset here to try out the algorithm yourself: https://drive.google.com/file/d/1o20lcmXU1HxOP4KsLXrXbjT2jTfjeaJh/view?usp=sharing

Inter-sensor rotation estimation

alt text

roslaunch linkalibr ros_calib_init.launch

I am working on making this code more generic, until then please take care to change the file path names.

Inter-sensor translation estimation

alt text

roslaunch linkalibr linkalibr_ouster_vectornav.launch

I am working on making this code more generic, until then please take care to change the file path names.

The results are stored in folder linkalibr/data as a homogenous transformation matrix in text file I_T_L_final.txt

Plots

We can plot the results of the EKF based estimation process by using the MATLAB plot files available in folder linkalibr/data.

The plot for calibration parameters is shown below:

Estimated inter-sensor translation parameter

alt text

Estimated Inter-sensor rotation parameter

alt text

Some more figures...

Estimated IMU Trajectory

alt text

Estimated IMU Velocity

alt text

Result of scan matching using motion compensated scans

alt text

Acknowledgement

This codebase was developed under the OpenVINS framework.

Comments
  • Segmentation fault (core dumped)

    Segmentation fault (core dumped)

    hello,I get an error when I run the program,Segmentation fault (core dumped).I pinpointed the cause of the error,in file ros_calib_init.cpp" LOdom = std::make_shared<lin_core::LidarOdometry>(ndt_resolution, "", true);".Do you know what caused it?Looking forward to your reply.

    opened by qingxiali 3
  • building error: undefined reference to symbol '_ZN3tbb8internal23allocate_via_handler_v3Em'

    building error: undefined reference to symbol '_ZN3tbb8internal23allocate_via_handler_v3Em'

    【Error log】 Errors << linkalibr:make /home/minzhus/receive/Lidar_imu_calib/TAMU/catkin_ws_ov/logs/linkalibr/build.make.001.log
    /usr/bin/ld: CMakeFiles/ros_calib_init_optimizer.dir/src/ros_calib_init_optimizer.cpp.o: undefined reference to symbol '_ZN3tbb8internal23allocate_via_handler_v3Em' //usr/lib/x86_64-linux-gnu/libtbb.so.2: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status make[2]: *** [/home/minzhus/receive/Lidar_imu_calib/TAMU/catkin_ws_ov/devel/.private/linkalibr/lib/linkalibr/ros_calib_init_optimizer] Error 1 make[1]: *** [CMakeFiles/ros_calib_init_optimizer.dir/all] Error 2 make: *** [all] Error 2

    【Prerequisites】: Ubuntu 16.04 system GTSAM - 4.0.3 openVINS-2.3 TBB: sudo apt-get install intel-tbb-2020.3-912

    【GTSAM Configuration Options】 -- ================ Configuration Options ====================== -- CMAKE_CXX_COMPILER_ID type : GNU -- CMAKE_CXX_COMPILER_VERSION : 5.5.0 -- CMake version : 3.18.5 -- CMake generator : Unix Makefiles -- CMake build tool : /usr/bin/make -- Build flags
    -- Build Tests : Enabled -- Build examples with 'make all' : Enabled -- Build timing scripts with 'make all': Disabled -- Build Docs : Enabled -- Build shared GTSAM libraries : Enabled -- Put build type in library name : Enabled -- Build libgtsam_unstable : Enabled -- Build for native architecture : Enabled -- Build type : Release -- C compilation flags : -O3 -DNDEBUG -- C++ compilation flags : -Wno-misleading-indentation -O3 -DNDEBUG -- GTSAM_COMPILE_FEATURES_PUBLIC : cxx_std_11 -- GTSAM_COMPILE_OPTIONS_PRIVATE : -Wall;-fPIC;$<$CONFIG:Debug:-g;-fno-inline>;$<$CONFIG:Release:-O3>;$<$CONFIG:Timing:-g;-O3>;$<$CONFIG:Profiling:-O3>;$<$CONFIG:RelWithDebInfo:-g;-O3>;-Wno-unused-local-typedefs -- GTSAM_COMPILE_OPTIONS_PUBLIC : -march=native -- GTSAM_COMPILE_DEFINITIONS_PRIVATE : $<$CONFIG:Debug:_DEBUG;EIGEN_INITIALIZE_MATRICES_BY_NAN>;$<$CONFIG:Release:NDEBUG>;$<$CONFIG:Timing:NDEBUG;ENABLE_TIMING>;$<$CONFIG:Profiling:NDEBUG>;$<$CONFIG:RelWithDebInfo:NDEBUG> -- GTSAM_COMPILE_DEFINITIONS_PUBLIC : -- GTSAM_COMPILE_OPTIONS_PRIVATE_DEBUG : -g;-fno-inline -- GTSAM_COMPILE_OPTIONS_PUBLIC_DEBUG : -- GTSAM_COMPILE_DEFINITIONS_PRIVATE_DEBUG : _DEBUG;EIGEN_INITIALIZE_MATRICES_BY_NAN -- GTSAM_COMPILE_DEFINITIONS_PUBLIC_DEBUG : -- GTSAM_COMPILE_OPTIONS_PRIVATE_RELEASE : -O3 -- GTSAM_COMPILE_OPTIONS_PUBLIC_RELEASE : -- GTSAM_COMPILE_DEFINITIONS_PRIVATE_RELEASE : NDEBUG -- GTSAM_COMPILE_DEFINITIONS_PUBLIC_RELEASE : -- GTSAM_COMPILE_OPTIONS_PRIVATE_TIMING : -g;-O3 -- GTSAM_COMPILE_OPTIONS_PUBLIC_TIMING : -- GTSAM_COMPILE_DEFINITIONS_PRIVATE_TIMING : NDEBUG;ENABLE_TIMING -- GTSAM_COMPILE_DEFINITIONS_PUBLIC_TIMING : -- GTSAM_COMPILE_OPTIONS_PRIVATE_PROFILING : -O3 -- GTSAM_COMPILE_OPTIONS_PUBLIC_PROFILING : -- GTSAM_COMPILE_DEFINITIONS_PRIVATE_PROFILING : NDEBUG -- GTSAM_COMPILE_DEFINITIONS_PUBLIC_PROFILING : -- GTSAM_COMPILE_OPTIONS_PRIVATE_RELWITHDEBINFO : -g;-O3 -- GTSAM_COMPILE_OPTIONS_PUBLIC_RELWITHDEBINFO : -- GTSAM_COMPILE_DEFINITIONS_PRIVATE_RELWITHDEBINFO : NDEBUG -- GTSAM_COMPILE_DEFINITIONS_PUBLIC_RELWITHDEBINFO : -- GTSAM_COMPILE_OPTIONS_PRIVATE_MINSIZEREL : -- GTSAM_COMPILE_OPTIONS_PUBLIC_MINSIZEREL : -- GTSAM_COMPILE_DEFINITIONS_PRIVATE_MINSIZEREL : -- GTSAM_COMPILE_DEFINITIONS_PUBLIC_MINSIZEREL : -- Use System Eigen : ON (Using version: 3.2.92) -- Use Intel TBB : Yes -- Eigen will use MKL : MKL not found -- Eigen will use MKL and OpenMP : OpenMP found but GTSAM_WITH_EIGEN_MKL is disabled -- Default allocator : TBB -- Cheirality exceptions enabled : YES -- Build with ccache : No -- Packaging flags
    -- CPack Source Generator : TGZ -- CPack Generator : TGZ -- GTSAM flags
    -- Quaternions as default Rot3 : Disabled -- Runtime consistency checking : Disabled -- Rot3 retract is full ExpMap : Enabled -- Pose3 retract is full ExpMap : Enabled -- Deprecated in GTSAM 4 allowed : Enabled -- Point3 is typedef to Vector3 : Disabled -- Metis-based Nested Dissection : Enabled -- Use tangent-space preintegration: Enabled -- Build Wrap : Enabled -- MATLAB toolbox flags
    -- Install MATLAB toolbox : Disabled -- Cython toolbox flags
    -- Install Cython toolbox : Disabled

    【Q&A】 What's the problem?How to resolve this error?

    opened by liang0724s 2
  • did it work betwwen external imu and lidar?

    did it work betwwen external imu and lidar?

    i just want to make sure that does it work between external imu and lidar, which get transform between lidar and external imu?or it only get the transform between ouster's internal imu and lidar?

    opened by Wuxinxiaoshifu 2
  • Compatible with livox lidars?

    Compatible with livox lidars?

    Hello guys,

    i would like to ask if your project is compatible with livox lidars.

    I have livox mid 40 and xsens mti g 710 gnss/ins and i want to calibrate them.

    Thanks in advance

    opened by ManChrys 1
  • Where to print the initilization result (rotation extrinsic)

    Where to print the initilization result (rotation extrinsic)

    Hi, your paper said an initialization module is implemented to obtain coarse rotation extrinsic, by aligning two rotation sequences (IMU angular velocity integration and NDT scan matching). But I didn't find where to print the initilization result, only final result in the end.

    Another question, do I need to give an initial guess of the extrinsic parameters manually? Because I print the result of

    Eigen::Matrix3d I_R_L = lin_core::quat_2_Rot(sys->get_state()->_calib_LIDARtoIMU->quat()); Eigen::Vector3d I_t_L = sys->get_state()->_calib_LIDARtoIMU->pos();

    I found the I_R_L just have a little change from default initial value (0,0,0) degree. Thanks!

    opened by zfc-zfc 1
  • Launch error

    Launch error

    Hi, I have successfully built the package and did the intrinsic calibration. I did change some of the requirements in the launch file to match up with my sensors. But I have an error when I tried to launch the package. I did catkin_make clean & change the permissions as well. But there's no difference.

    ERROR: cannot launch node of type [linkalibr/ros_calib_init]: Cannot locate node of type [ros_calib_init] in package [linkalibr]. Make sure file exists in package path and permission is set to executable (chmod +x) ERROR: cannot launch node of type [linkalibr/ros_calib_init_optimizer]: Cannot locate node of type [ros_calib_init_optimizer] in package [linkalibr]. Make sure file exists in package path and permission is set to executable (chmod +x)

    opened by SyahirMuzni 5
  • Could I use this tool on the Ground robot dataset?

    Could I use this tool on the Ground robot dataset?

    @SubMishMar
    Thanks for your great work.

    I have a question.

    I want to know the extrinsic calibration of Velodyne VLP 16 and Bosch BMI055(built-in IMU on realsense D435i).

    Since it is attached to the ground robot, so it is difficult to obtain the same motion as the Data collection procedure video.

    Can I use this tool to get an extrinsic parameter with the dataset I got from moving the Ground robot?

    Thanks,

    opened by Taeyoung96 12
  • show the process to get the Hx

    show the process to get the Hx

    Thanks for the opening code. I have read the source code and use this code in my experiments. Can you show me how you get the H1_xi_rot, H1_Xj_rot in function updateScan2Scan, and H2_xc_rot in updateScan2GlobalMap?

    opened by WeichuanPan 0
  • REQUIRED process [ros_calib_init_optimizer-2] has died!

    REQUIRED process [ros_calib_init_optimizer-2] has died!

    运行roslaunch linkalibr ros_calib_init.launch后报错如下:

    [ INFO] [1652091019.397516439]: Frame: 547 / 550 [ INFO] [1652091019.652656943]: Frame: 548 / 550 [ INFO] [1652091020.011336717]: Frame: 549 / 550 [ INFO] [1652091020.301443733]: Frame: 550 / 550 Rot3:

    0.999763 0.0107452 0.0189149 0.0106649 -0.999934 0.00433886 0.0189603 -0.00413611 -0.999812 Euler Angles: 0.248644 178.916 179.384 Marginal Covariance 0.596503 -0.00196232 -0.00136384 -0.00196232 0.587782 0.0279263 -0.00136384 0.0279263 0.594626 ================================================================================REQUIRED process [ros_calib_init_optimizer-2] has died! process has finished cleanly log file: /home/hqc/.ros/log/3c887e46-cf4f-11ec-aca0-9822ef00c7d8/ros_calib_init_optimizer-2*.log Initiating shutdown!

    [ros_calib_init_optimizer-2] killing on exit [ros_calib_init-1] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done

    opened by huanghqc 2
  • About timestamp synchronization

    About timestamp synchronization

    In ros_calib_init.cpp,

        /// Handle Lidar measurement
        sensor_msgs::PointCloud2::ConstPtr s_lidar = m.instantiate<sensor_msgs::PointCloud2>();
        if (s_lidar != nullptr && m.getTopic() == topic_lidar) {
            lin_core::VPointCloud::Ptr cloud_pcl(new lin_core::VPointCloud);
            pcl::fromROSMsg(*s_lidar, *cloud_pcl);
            LOdom->feedScan((*s_lidar).header.stamp.toSec(), cloud_pcl);
            LOdom->append_and_update(true);
            Eigen::Matrix3d deltaR_L = LOdom->get_latest_relativePose().odometry_ij.block(0, 0, 3, 3);
            Eigen::Quaterniond delta_qL(deltaR_L);
            geometry_msgs::PoseStamped pose;
            pose.header.stamp = s_lidar->header.stamp;
            pose.pose.orientation.x = delta_qL.x();
            pose.pose.orientation.y = delta_qL.y();
            pose.pose.orientation.z = delta_qL.z();
            pose.pose.orientation.w = delta_qL.w();
            lodom_publisher.publish(pose);
            **imupacket.header.stamp = s_lidar->header.stamp;**
            imu_packet_publisher.publish(imupacket);
            imupacket.stamps.clear();
            imupacket.accelreadings.clear();
            imupacket.gyroreadings.clear();
        }
    

    I don't understand why "imupacket.header.stamp = s_lidar->header.stamp". Why not use the timestamp of IMU?

    opened by codehory 3
  • In your paper, how do you get formula 3 from formula 2?

    In your paper, how do you get formula 3 from formula 2?

    Hi~ First of all, thank you for your contribution. When I read your paper, I didn't understand how to get formula 3 from formula 2. Can you tell me? thanks!

    opened by 772894624 5
Owner
Unmanned Systems Lab
Unmanned Systems Lab at Texas A&M
Unmanned Systems Lab
Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X

Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X (probably BSD also). Web Link Website

Arun Narayanankutty 169 Nov 20, 2022
Vizzu is a free, open-source Javascript/C++ library for animated data visualizations and data stories.

Vizzu is a free, open-source Javascript/C++ library utilizing a generic dataviz engine that generates many types of charts and seamlessly animates between them

Vizzu 1.6k Dec 1, 2022
A set of open c++ game development tools that are lightweight, easy-to-integrate and free to use. Currently hosting a magicavoxel .vox full scene loader.

open game tools Open game tools is a set of unencumbered, free, lightweight, easy-to-integrate tools for use in game development. So far it contains:

null 288 Nov 26, 2022
A completely free, open-source, 2D game engine built on proven torque technology.

Torque2D 4.0 Early Access 1 MIT Licensed Open Source version of Torque2D from GarageGames. Maintained by the Torque Game Engines team and contribution

Torque Game Engines 652 Dec 3, 2022
Dear ImGui is a bloat-free graphical user interface library for C++

dear imgui (This library is available under a free and permissive license, but needs financial support to sustain its continued improvements. In addit

Douglas McCloskey 6 Oct 27, 2020
Source Code for "Ray Tracing Gems: High-Quality and Real-Time Rendering with DXR and Other APIs" by Eric Haines and Tomas Akenine-Möller

Apress Source Code This repository accompanies Ray Tracing Gems: High-Quality and Real-Time Rendering with DXR and Other APIs by Eric Haines and Tomas

Apress 853 Dec 2, 2022
Horde3D is a small 3D rendering and animation engine. It is written in an effort to create an engine being as lightweight and conceptually clean as possible.

Horde3D Horde3D is a 3D rendering engine written in C++ with an effort being as lightweight and conceptually clean as possible. Horde3D requires a ful

Volker Vogelhuber 1.3k Nov 28, 2022
Lightweight and modular C++11 graphics middleware for games and data visualization

Magnum — Lightweight and modular C++11/C++14 graphics middleware for games and data visualization Looking for an open-source library that gives you gr

Vladimír Vondruš 4.3k Nov 28, 2022
ANSI C library for NURBS, B-Splines, and Bézier curves with interfaces for C++, C#, D, Go, Java, Lua, Octave, PHP, Python, R, and Ruby.

TinySpline TinySpline is a small, yet powerful library for interpolating, transforming, and querying arbitrary NURBS, B-Splines, and Bézier curves. Th

Marcel Steinbeck 876 Nov 26, 2022
Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Raul Mur-Artal 7.8k Dec 5, 2022
A C++/DirectX 11 implementation of "A Scalable and Production Ready Sky and Atmosphere Rendering Technique"

Atmosphere Renderer A C++/DirectX 11 implementation of "A Scalable and Production Ready Sky and Atmosphere Rendering Technique" Features interactive e

Z Guan 37 Nov 20, 2022
StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR!

StereoKit is an easy-to-use open source mixed reality library for building HoloLens and VR applications with C# and OpenXR! Inspired by libraries like XNA and Processing, StereoKit is meant to be fun to use and easy to develop with, yet still quite capable of creating professional and business ready software.

Nick Klingensmith 720 Dec 3, 2022
FFVideo - an example FFmpeg lib, and wxWidgets Player with video filters and face detection

An example FFmpeg lib, and wxWidgets Player application with video filters and face detection, it is a no-audio video player intended for video experiments and developers learning how to code media applications.

Blake Senftner 20 Mar 26, 2022
Draco is a library for compressing and decompressing 3D geometric meshes and point clouds.

Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.

Google 5.3k Nov 27, 2022
A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

null 2.4k Dec 4, 2022
Brand new engine with new and QoL features. Grafex is Psych engine with some additions and Better graphics

Friday Night Funkin' - Graphex Engine Credits: Grafex Mod aka Psych Graphic Rework: Xale - Lead Coding, Artist PurpleSnake - Second Coder Psych Engine

Xale 3 Sep 5, 2022
Powerful, easy to use, and portable visualization toolkit for mixed 3D and 2D content

Powerful, easy to use, and portable visualization toolkit for mixed 3D and 2D content

Microsoft 134 Nov 29, 2022
Efficiently spawn and move high amounts of objects like bullets for bullet hells, particles and more.

Godot Native Bullets Efficiently spawn and move high amounts of objects like bullets for bullet hells, particles and more. This is a GDNative plugin,

Samuele Zolfanelli 99 Nov 20, 2022
A very simple and light-weight drawing app made with qt and C++.

Blackboard A very simple and light-weight drawing app made with qt and C++. It supports tablet and pen pressure with the help of QTabletEvents. So you

null 1 Nov 15, 2021