VLOAM-CMU-16833 - CMU 16-833 "Robot Localization and Mapping" Course Project

Overview

Introduction

This repository is a reimplementation of the VLOAM algorithm [1]. The LOAM/Lidar Odometry part is adapted and refactored from ALOAM [2], and the Visual Odometry part is written according to the DEMO paper [3].

The following figure [1] illustrates the pipeline of the VLOAM algorithm.

demo

Results

demo

Video: https://youtu.be/NnoxB3r_cDM

demo

Detailed Usage

Check README.md under src/vloam_main

Prerequisites

OpenCV 4.5.1 Eigen3 3.3 Ceres 2.0 PCL 1.2

Evaluation tool

demo

https://github.com/LeoQLi/KITTI_odometry_evaluation_tool

Data format

Place bag files under "src/vloam_main/bags/"

Note: current dataloader only support "synced" type dataset.

Reference:

[1] J. Zhang and S. Singh. Laser-visual-inertial Odometry and Mapping with High Robustness and Low Drift. Journal of Field Robotics. vol. 35, no. 8, pp. 1242–1264, 2018.

[2] T. Qin and S. Cao. A-LOAM. https://github.com/HKUST-Aerial-Robotics/A-LOAM

[3] Zhang, Ji, Michael Kaess, and Sanjiv Singh. "Real-time depth enhanced monocular odometry." 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014.

Comments
  • OPENCV Problems?

    OPENCV Problems?

    Hello,I meet this problems when catkin_make , where is the problems? what should we do ?

    problems: /catkin_ws/devel/lib/libimage_util.so:对‘cv::imshow(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, cv::_InputArray const&)’未定义的引用 /catkin_ws/devel/lib/libimage_util.so:对‘cv::namedWindow(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int)’未定义的引用 ....

    opened by firstbeer 12
  • Linking error to OpenCV

    Linking error to OpenCV

    I am getting a linking error to OpenCV in catkin_make. I am using ros melodic and Ubuntu 18.04.5. Have you faced this issue?

    [ 92%] Built target laser_mapping [ 94%] Building CXX object VLOAM-CMU-16833/src/lidar_odometry_mapping/CMakeFiles/lidar_odometry_mapping.dir/src/lidar_odometry_mapping.cpp.o [ 96%] Linking CXX shared library /home/beastv2/catkin_ws/devel/lib/liblidar_odometry_mapping.so [ 96%] Built target lidar_odometry_mapping [ 98%] Building CXX object VLOAM-CMU-16833/src/vloam_main/CMakeFiles/vloam_main_node.dir/src/vloam_main_node.cpp.o [100%] Linking CXX executable /home/beastv2/catkin_ws/devel/lib/vloam_main/vloam_main_node /home/beastv2/catkin_ws/devel/lib/libimage_util.so: undefined reference to cv::imshow(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cv::_InputArray const&)' /home/beastv2/catkin_ws/devel/lib/libimage_util.so: undefined reference tocv::namedWindow(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, int)' /home/beastv2/catkin_ws/devel/lib/libimage_util.so: undefined reference to cv::calcOpticalFlowPyrLK(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::_InputOutputArray const&, cv::_OutputArray const&, cv::_OutputArray const&, cv::Size_<int>, int, cv::TermCriteria, int, double)' /home/beastv2/catkin_ws/devel/lib/libimage_util.so: undefined reference tocv::waitKey(int)' /home/beastv2/catkin_ws/devel/lib/libpoint_cloud_util.so: undefined reference to `cv::setMouseCallback(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, void ()(int, int, int, int, void), void*)' collect2: error: ld returned 1 exit status VLOAM-CMU-16833/src/vloam_main/CMakeFiles/vloam_main_node.dir/build.make:975: recipe for target '/home/beastv2/catkin_ws/devel/lib/vloam_main/vloam_main_node' failed make[2]: *** [/home/beastv2/catkin_ws/devel/lib/vloam_main/vloam_main_node] Error 1 CMakeFiles/Makefile2:2542: recipe for target 'VLOAM-CMU-16833/src/vloam_main/CMakeFiles/vloam_main_node.dir/all' failed make[1]: *** [VLOAM-CMU-16833/src/vloam_main/CMakeFiles/vloam_main_node.dir/all] Error 2 Makefile:145: recipe for target 'all' failed make: *** [all] Error 2 Invoking "make -j8 -l8" failed

    opened by kloukiniotisandreas 11
  • Unable to obtain the results

    Unable to obtain the results

    Hi,

    I am having a little trouble obtaining the results from this package. I have saved my kitti rosbag files in /VLOAM_ws/src/vloam_main/bags. I ran the command roslaunch vloam_main vloam_main.launch which opens rviz and then in a separate terminal I ran my rosbag files.

    At this point should I see any results in rviz? At the end of the rosbag file I checked whether any results are saved in /VLOAM_ws/src/vloam_main/results but there were none even though I have made save_traj, true. I am not sure whether I am missing something here.

    Any help is much appreciated.

    opened by dilaxy123 7
  • real time slam

    real time slam

    hello, after i changed the code to do real slam(haven't calibrate radar and camera), when i launch the file, the following error occurred, do you have any suggestions? I have no idea about the imu_link, i only find it in .rviz file

    image

    opened by arashizn 4
  • 'process had died' problem

    'process had died' problem

    Hi.

    I am having a problem when I try step4 in this page https://github.com/YukunXia/VLOAM-CMU-16833/tree/master/src/vloam_main#usage-1

    I follow this page and use roptopic pub. But, my roslaunch terminal showed error "[vloam_main_node-1] process has died [pid 4477, exit code -11, .."

    I am not sure whether I am missing something here. Any help is much appreciated.

    opened by ehdwp 4
  • Error in cv

    Error in cv

    I have OpenCV 4.5.3 but I am getting the following errors when I tried to build the package. Is it because SIFT is no longer available in OpenCV? (for the second error)

    /home/didula/BVLOAM_ws/src/visual_odometry/src/image_util.cpp: In member function ‘std::vectorcv::KeyPoint vloam::ImageUtil::detKeypoints(cv::Mat&)’: /home/didula/BVLOAM_ws/src/visual_odometry/src/image_util.cpp:56:7: error: ‘ScoreType’ is not a member of ‘cv::ORB’ cv::ORB::ScoreType scoreType = cv::ORB::FAST_SCORE; ^ /home/didula/BVLOAM_ws/src/visual_odometry/src/image_util.cpp:59:104: error: ‘scoreType’ was not declared in this scope detector = cv::ORB::create(num_features, scaleFactor, nlevels, edgeThreshold, firstLevel, WTA_K, scoreType, ^ /home/didula/BVLOAM_ws/src/visual_odometry/src/image_util.cpp:65:22: error: ‘cv::SIFT’ has not been declared detector = cv::SIFT::create(); ^ /home/didula/BVLOAM_ws/src/visual_odometry/src/image_util.cpp: In member function ‘cv::Mat vloam::ImageUtil::descKeypoints(std::vectorcv::KeyPoint&, cv::Mat&)’: /home/didula/BVLOAM_ws/src/visual_odometry/src/image_util.cpp:192:21: error: ‘cv::SIFT’ has not been declared extractor = cv::SIFT::create(); ^ visual_odometry/CMakeFiles/image_util.dir/build.make:62: recipe for target 'visual_odometry/CMakeFiles/image_util.dir/src/image_util.cpp.o' failed make[2]: *** [visual_odometry/CMakeFiles/image_util.dir/src/image_util.cpp.o] Error 1 CMakeFiles/Makefile2:2626: recipe for target 'visual_odometry/CMakeFiles/image_util.dir/all' failed make[1]: *** [visual_odometry/CMakeFiles/image_util.dir/all] Error 2

    opened by didzdissanayaka8 3
  • Unable to run rosbag file

    Unable to run rosbag file

    Hello Yukun, Thank you for such a great implementation. My team and I have been trying to implement your repo but are running into the following issue. Could you please guide us in any possible way?

    Thank you!

    IMG-1357

    opened by parthmalpathak 2
  • Some problem about the Eigen

    Some problem about the Eigen

    When I launch the vloam and public the topic to run the bag file, the error appear. `[email protected]:~/catkin/vloam/VLOAM-NOTED-master$ roslaunch vloam_main vloam_main.launch ... logging to /home/ysy/.ros/log/c999d416-6397-11ec-adfc-200db02d4198/roslaunch-ysy-To-be-filled-by-O-E-M-5684.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

    started roslaunch server http://ysy-To-be-filled-by-O-E-M:37497/

    SUMMARY

    PARAMETERS

    • /CLAHE: False
    • /detach_VO_LO: True
    • /keypoint_NMS: False
    • /loam_verbose_level: 1
    • /map_pub_number: 20
    • /mapping_line_resolution: 0.4
    • /mapping_plane_resolution: 0.8
    • /mapping_skip_frame: 1
    • /minimum_range: 5.0
    • /optical_flow_match: False
    • /remove_VO_outlier: 100
    • /reset_VO_to_identity: False
    • /rosdistro: melodic
    • /rosversion: 1.14.12
    • /scan_line: 64
    • /visualize_optical_flow: True
    • /vloam_main_node/publish_point_cloud: True
    • /vloam_main_node/rosbag_rate: 0.33
    • /vloam_main_node/save_traj: True
    • /vloam_main_node/visualize_depth: False

    NODES / rviz (rviz/rviz) vloam_main_node (vloam_main/vloam_main_node)

    ROS_MASTER_URI=http://localhost:11311

    process[vloam_main_node-1]: started with pid [5699] process[rviz-2]: started with pid [5700] [ WARN] [1640231903.940193533]: Shutdown request received. [ WARN] [1640231903.940258711]: Reason given for shutdown: [[/rviz] Reason: new node registered with same name] [ INFO] [1640231920.634819865]: LO file path; /home/ysy/catkin/vloam/VLOAM-NOTED-master/src/vloam_main/results/2011_09_30_drive_0020/LO1.txt

    vloam_main_node: /usr/include/eigen3/Eigen/src/Core/DenseStorage.h:128: Eigen::internal::plain_array<T, Size, MatrixOrArrayOptions, 32>::plain_array() [with T = float; int Size = 16; int MatrixOrArrayOptions = 0]: Assertion (internal::UIntPtr(eigen_unaligned_array_assert_workaround_gcc47(array)) & (31)) == 0 && "this assertion is explained here: " "http://eigen.tuxfamily.org/dox-devel/group__TopicUnalignedArrayAssert.html" " **** READ THIS WEB PAGE !!! ****"' failed. [vloam_main_node-1] process has died [pid 5699, exit code -6, cmd /home/ysy/catkin/vloam/VLOAM-NOTED-master/devel/lib/vloam_main/vloam_main_node __name:=vloam_main_node __log:=/home/ysy/.ros/log/c999d416-6397-11ec-adfc-200db02d4198/vloam_main_node-1.log]. log file: /home/ysy/.ros/log/c999d416-6397-11ec-adfc-200db02d4198/vloam_main_node-1*.log Can you give me some advice how to fix it? Thanks for your reply.

    opened by onionysy 2
  • Dependency problem

    Dependency problem

    Hi, I have OpenCV 4.5.1 installed and when I catkin_make I get the following error. May I know what ubuntu version you used for the original version?

    make[2]: *** No rule to make target '/usr/lib/x86_64-linux-gnu/libconsole_bridge.soopencv_ml', needed by '/home/dilaxy/BVLOAM_ws/devel/lib/libimage_util.so'. Stop. CMakeFiles/Makefile2:2626: recipe for target 'visual_odometry/CMakeFiles/image_util.dir/all' failed make[1]: *** [visual_odometry/CMakeFiles/image_util.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [ 3%] Built target vloam_tf make[2]: *** No rule to make target '/usr/lib/x86_64-linux-gnu/libconsole_bridge.soopencv_ml', needed by '/home/dilaxy/BVLOAM_ws/devel/lib/libpoint_cloud_util.so'. Stop. CMakeFiles/Makefile2:2702: recipe for target 'visual_odometry/CMakeFiles/point_cloud_util.dir/all' failed make[1]: *** [visual_odometry/CMakeFiles/point_cloud_util.dir/all] Error 2

    opened by dilaxy123 2
  • Error: Tried to advertise a service that is already advertised

    Error: Tried to advertise a service that is already advertised

    Thanks. when I roslaunch this, process[vloam_main_node -1]: started with pid [18081] process[rviz-2]: started with pid [18082] [ERROR] [1626056112.350555675]: Tried to advertise a service that is already advertised in this node [/rviz/compressed/setparameters] There is nothing when I play the bag.

    Originally posted by @wwq985 in https://github.com/YukunXia/VLOAM-CMU-16833/issues/4#issuecomment-877922595

    @wwq985 Please post all your running steps, and the output screenshots here. Also, please attach your system information, eg. which OS and which version.

    opened by YukunXia 2
  • A question about

    A question about "demo_lidar"!

    I'm sorry to bother you. l have a problem which I can't run the "demo_lidar" with kitti. Do you know how to run it with the kitti. I will appreciate if you can repply to me.

    opened by KalmanSLAMer 1
Owner
Yukun Xia
Master in Robotic Systems Development
Yukun Xia
Assignment skeleton for course on Discrete Differential Geometry (15-458/858)

ddg-exercises This repo contains C++ skeleton code for course assignments from Discrete Differential Geometry (15-458/858). For the JavaScript version

Geometry Collective 86 Sep 30, 2022
Ebox questions for UTA003 course

e-box-UTA003 Stuck on Ebox? We have Ebox coding questions for UTA003 course. Made by a bunch of students from CSE 20-24 batch. Contributions to this r

null 20 Jul 7, 2022
Repository for course content (homeworks, projects, recitation exercises) of CSCI 1300 in Spring 2022.

CSCI1300 Spring 2022 Repository for course content (e.g. homework assignments, project write-ups, recitation exercises) of CSCI 1300 in Spring 2022. C

null 12 Apr 6, 2022
kaldi-asr/kaldi is the official location of the Kaldi project.

Kaldi Speech Recognition Toolkit To build the toolkit: see ./INSTALL. These instructions are valid for UNIX systems including various flavors of Linux

Kaldi 12k Oct 5, 2022
This fork adds enhancements to the Loz project (Legend of Zelda remake).

LOZ This project is a remake of the game Legend of Zelda. Summary The repository is split into a game project and two tools. The tools extract resourc

Aldo Núñez 24 Aug 28, 2022
this is a repo of where you will get to see tic tac toe AI intregrated project

?? Tic-Tac-Toe-AI-Intregrated ?? What is the meaning of AI Intregrated ??‍♀️ ??‍♂️ ❓ ❓ You all have Played Tic Tac Toe in your life if you don't know

Ujjwal 16 Sep 11, 2022
This is a small example project, that showcases the possibility of using a surrogate model to estimate the drag coefficient of arbitrary triangles.

flowAroundTriangles This is a small example project, that showcases the possibility of using a surrogate model to estimate the drag coefficient of arb

null 6 Sep 16, 2022
A project to control Petoi Bittle using human pose

Petoi Bittle Controlled by Pose A project to control Petoi Bittle by human pose Human pose is estimated using MoveNet and TensorFlow Lite YouTube Syst

iwatake 11 Dec 26, 2021
This is a sample ncnn android project, it depends on ncnn library and opencv

This is a sample ncnn android project, it depends on ncnn library and opencv

null 231 Sep 30, 2022
A project demonstration on how to use the GigE camera to do the DeepStream Yolo3 object detection

A project demonstration on how to use the GigE camera to do the DeepStream Yolo3 object detection, how to set up the GigE camera, and deployment for the DeepStream apps.

NVIDIA AI IOT 9 Sep 23, 2022
OpenSpeaker is a completely independent and open source speaker recognition project.

OpenSpeaker is a completely independent and open source speaker recognition project. It provides the entire process of speaker recognition including multi-platform deployment and model optimization.

ZY 33 Sep 15, 2022
Mirror of compiler project code. Not for SVC purpose.

Compiler-proj Project progress is updated here. Progress 2021/11/28: Started! Set up Makefile and finished basic scanner. 2021/10/24: Repo created. Ac

Yuheng 0 Dec 23, 2021
This is my first 42 cursus project. I'm going to remake some built-in c functions so I can reuse them.

lib-ft This is my first 42 cursus project. I'm going to remake some built-in c functions so I can reuse them in my cursus. "Read the subject.pdf for m

Mhamed Ajjig 4 Sep 3, 2022
This is my first 42 cursus project. I'm going to remake some built-in c functions so I can reuse them.

lib-ft This is my first 42 cursus project. I'm going to remake some built-in c functions so I can reuse them in my cursus. "Read the subject.pdf for m

Mhamed Ajjig 4 Dec 17, 2021
The purpose of this project is to apply mediapipe to more AI chips.

1.About This Project Our Official Website: www.houmo.ai Who We Are: We are Houmo - A Great AI Company. We wish to change the world with unlimited comp

null 36 Aug 29, 2022
BladeDISC - BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

BladeDISC Introduction Overview Features and Roadmap Frontend Framework Support Matrix Backend Support Matrix Deployment Solutions Numbers of Typical

Alibaba 456 Sep 28, 2022
UVIC ECE 499 Real-Time Gesture Recognition Project

Welcome to GitHub Pages You can use the editor on GitHub to maintain and preview the content for your website in Markdown files. Whenever you commit t

Lyndon Bauto 3 Sep 21, 2019
Database system project based on CMU 15-445/645 (FALL 2020)

Database system project based on CMU 15-445/645 (FALL 2020)

null 21 Sep 27, 2022
Project 1945 is a top down scroller game made with SDL2, CMake and programmed in C as a project of study for the Italian Videogames Academy for the 3rd year course of Videogames programming.

Project 1945 is a top down scroller game made with SDL2, CMake and programmed in C as a project of study for the Italian Videogames Academy for the 3rd year course of Videogames programming. The game is based on the old videogame 1945: The Final Front of the 2002.

null 18 Jul 27, 2022