ICRA 2021 - Robust Place Recognition using an Imaging Lidar

Overview

Robust Place Recognition using an Imaging Lidar

A place recognition package using high-resolution imaging lidar. For best performance, a lidar equipped with more than 64 uniformly distributed channels is strongly recommended, i.e., Ouster OS1-128 lidar.


Dependency

  • ROS
  • DBoW3
    cd ~/Downloads/
    git clone https://github.com/rmsalinas/DBow3.git
    cd ~/Downloads/DBow3/
    mkdir build && cd build
    cmake -DCMAKE_BUILD_TYPE=Release ..
    sudo make install
    

Install Package

Use the following commands to download and compile the package.

cd ~/catkin_ws/src
git clone https://github.com/TixiaoShan/imaging_lidar_place_recognition.git
cd ..
catkin_make

Notes

Download

The three datasets used in the paper can be downloaded from from Google Drive. The lidar used for data-gathering is Ouster OS1-128.

https://drive.google.com/drive/folders/1G1kE8oYGKj7EMdjx7muGucXkt78cfKKU?usp=sharing

Point Cloud Format

The author defined a customized point cloud format, PointOuster, in parameters.h. The customized point cloud is projected onto various images in image_handler.h. If you are using your own dataset, please modify these two files to accommodate data format changes.

Visualization logic

In the current implementation, the package subscribes to a path message that is published by a SLAM framework, i.e., LIO-SAM. When a new point cloud arrives, the package associates the point cloud with the latest pose in the path. If a match is detected between two point clouds, an edge marker is plotted between these two poses. The reason why it's implemented in this way is that SLAM methods usually suffer from drift. If a loop-closure is performed, the associated pose of a point cloud also needs to be updated. Thus, this visualization logic can update point clouds using the updated path rather than using TF or odometry that cannot be updated later.

Image Crop

It's recommended to set the image_crop parameter in params.yaml to be 196-256 when testing the indoor and handheld datasets. This is because the operator is right behind the lidar during the data-gathering process. Using features extracted from the operator body may cause unreliable matching. This parameter should be set to 0 when testing the Jackal dataset, which improves the reverse visiting detection performance.


Test Package

  1. Run the launch file:
roslaunch imaging_lidar_place_recognition run.launch
  1. Play existing bag files:
rosbag play indoor_registered.bag -r 3

Paper

Thank you for citing our paper if you use any of this code or datasets.

@inproceedings{robust2021shan,
  title={Robust Place Recognition using an Imaging Lidar},
  author={Shan, Tixiao and Englot, Brendan and Duarte, Fabio and Ratti, Carlo and Rus Daniela},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  pages={to-be-added},
  year={2021},
  organization={IEEE}
}

Acknowledgement

  • The point clouds in the provided datasets are registered using LIO-SAM.
  • The package is heavily adapted from Vins-Mono.
You might also like...
OpenSpeaker is a completely independent and open source speaker recognition project.

OpenSpeaker is a completely independent and open source speaker recognition project. It provides the entire process of speaker recognition including multi-platform deployment and model optimization.

Strong Othello AI based on Egaroucid4, which got 1st place in the world

Egaroucid5 Strong Othello AI based on Egaroucid4, which got 1st place in the world Abstract Egaroucid5 is an Othello AI. You can play light version of

Very portable voice recorder with speech recognition.

DictoFun Small wearable voice recorder. NRF52832-based. Concept Device was initiated after my frustration while using voice recorder for storing ideas

UVIC ECE 499 Real-Time Gesture Recognition Project

Welcome to GitHub Pages You can use the editor on GitHub to maintain and preview the content for your website in Markdown files. Whenever you commit t

PocketSphinx is a lightweight speech recognition engine, specifically tuned for handheld and mobile devices, though it works equally well on the desktop

PocketSphinx 5prealpha This is PocketSphinx, one of Carnegie Mellon University's open source large vocabulary, speaker-independent continuous speech r

Official page of
Official page of "Patchwork: Concentric Zone-based Region-wise Ground Segmentation with Ground Likelihood Estimation Using a 3D LiDAR Sensor"

Patchwork Official page of "Patchwork: Concentric Zone-based Region-wise Ground Segmentation with Ground Likelihood Estimation Using a 3D LiDAR Sensor

IA-LIO-SAM is enhanced LIO-SAM using Intensity and Ambient channel from OUSTER LiDAR.
IA-LIO-SAM is enhanced LIO-SAM using Intensity and Ambient channel from OUSTER LiDAR.

IA-LIO-SAM Construction monitoring is one of the key modules in smart construction. Unlike structured urban environment, construction site mapping is

 LiDAR-Camera Calibration using 3D-3D Point correspondences
LiDAR-Camera Calibration using 3D-3D Point correspondences

ROS package to find a rigid-body transformation between a LiDAR and a camera for "LiDAR-Camera Calibration using 3D-3D Point correspondences"

3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint
3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint

3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint

Comments
  • Reverse visiting detection performance in KITTI dataset 08

    Reverse visiting detection performance in KITTI dataset 08

    Hi, @TixiaoShan Thanks for sharing your great work, I have test the method using KITTI dataset. The results show that reverse visiting detection performance is not well while the forward direction is pretty good, do you have any ideas about improving the reverse visiting recall?

    Screenshot from 2021-04-23 16-02-10 Screenshot from 2021-04-23 15-55-25

    opened by kinggreat24 6
  • Process died

    Process died

    Hello, thank you for sharing you work. When I run the launch file, it show

    [imaging_lidar_place_recognition_main-2] process has died [pid 20188, exit code -11, cmd /home/yunqi/snap/SLAM/loop_closure/Imaging/devel/lib/imaging_lidar_place_recognition/imaging_lidar_place_recognition_main __name:=imaging_lidar_place_recognition_main __log:=/home/yunqi/.ros/log/d43f6938-7f32-11ec-a38d-a87d12044727/imaging_lidar_place_recognition_main-2.log]. log file: /home/yunqi/.ros/log/d43f6938-7f32-11ec-a38d-a87d12044727/imaging_lidar_place_recognition_main-2*.log

    How can I solve it? One possible reason might be the version of OpenCV. I compile with openCV4 and it shows some warning

    /usr/bin/ld: warning: libopencv_video.so.3.2, needed by /usr/lib/x86_64-linux-gnu/libopencv_shape.so, may conflict with libopencv_video.so.4.2 /usr/bin/ld: warning: libopencv_videoio.so.3.2, needed by /usr/lib/x86_64-linux-gnu/libopencv_superres.so, may conflict with libopencv_videoio.so.4.2 /usr/bin/ld: warning: libopencv_photo.so.3.2, needed by /usr/lib/x86_64-linux-gnu/libopencv_videostab.so, may conflict with libopencv_photo.so.4.2 /usr/bin/ld: warning: libopencv_calib3d.so.3.2, needed by /usr/lib/x86_64-linux-gnu/libopencv_videostab.so, may conflict with libopencv_calib3d.so.4.2 /usr/bin/ld: warning: libopencv_features2d.so.3.2, needed by /usr/lib/x86_64-linux-gnu/libopencv_videostab.so, may conflict with libopencv_features2d.so.4.2 /usr/bin/ld: warning: libopencv_dnn.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_dnn.so.4.2 /usr/bin/ld: warning: libopencv_highgui.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_highgui.so.4.2 /usr/bin/ld: warning: libopencv_ml.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_ml.so.4.2 /usr/bin/ld: warning: libopencv_objdetect.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_objdetect.so.4.2 /usr/bin/ld: warning: libopencv_shape.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_shape.so.3.2 /usr/bin/ld: warning: libopencv_shape.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_shape.so.3.2 /usr/bin/ld: warning: libopencv_stitching.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_stitching.so.4.2 /usr/bin/ld: warning: libopencv_superres.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_superres.so.3.2 /usr/bin/ld: warning: libopencv_superres.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_superres.so.3.2 /usr/bin/ld: warning: libopencv_videostab.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_videostab.so.3.2 /usr/bin/ld: warning: libopencv_videostab.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_videostab.so.3.2 /usr/bin/ld: warning: libopencv_viz.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_viz.so.3.2 /usr/bin/ld: warning: libopencv_viz.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_viz.so.3.2 /usr/bin/ld: warning: libopencv_calib3d.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_calib3d.so.4.2 /usr/bin/ld: warning: libopencv_features2d.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_features2d.so.4.2 /usr/bin/ld: warning: libopencv_flann.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_flann.so.4.2 /usr/bin/ld: warning: libopencv_photo.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_photo.so.4.2 /usr/bin/ld: warning: libopencv_video.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_video.so.4.2 /usr/bin/ld: warning: libopencv_videoio.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_videoio.so.4.2 /usr/bin/ld: warning: libopencv_imgcodecs.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_imgcodecs.so.3.2 /usr/bin/ld: warning: libopencv_imgcodecs.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_imgcodecs.so.4.2 /usr/bin/ld: warning: libopencv_imgproc.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_imgproc.so.3.2 /usr/bin/ld: warning: libopencv_imgproc.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_imgproc.so.4.2 /usr/bin/ld: warning: libopencv_core.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_core.so.3.2 /usr/bin/ld: warning: libopencv_core.so.3.4, needed by /usr/local/lib/gcc/x86_64-linux-gnu/9.2.0/../../../libDBoW3.so, may conflict with libopencv_core.so.4.2 /usr/bin/ld: warning: libopencv_flann.so.3.2, needed by //usr/lib/x86_64-linux-gnu/libopencv_calib3d.so.3.2, may conflict with libopencv_flann.so.4.2

    What's your version of openCV

    opened by smalltheater 1
  • Reproducing intensity image on indoor dataset

    Reproducing intensity image on indoor dataset

    I am trying to extract the intensity image of the indoor dataset, but am running into issues.

    This is the result of running your code on the indoor dataset

    This is the result of running your code on your rooftoop dataset (from LIO-SAM), which looks great to me! That bagfile does not include ring, time or noise data, however. Must be an ouster driver issue?

    Any ideas where the problem could be? Thanks.

    opened by juliangaal 0
  • Point Cloud Format question

    Point Cloud Format question

    Hi,author, Thanks for sharing your great work! I have adjust the params.yaml to fit my OS1-64 laser,image_width: 1024, image_height: 64 and the cloud_topic, But I can't get a good image_intensity like your demo or parper, It looks like there is a offset in different rows. Is your data preprocessed ? Or there is something wrong with my params.yaml? image thanks for your answering!

    opened by chengwei0427 0
Gesture Recognition Toolkit (GRT) is a cross-platform, open-source, C++ machine learning library designed for real-time gesture recognition.

Gesture Recognition Toolkit (GRT) The Gesture Recognition Toolkit (GRT) is a cross-platform, open-source, C++ machine learning library designed for re

Nicholas Gillian 793 Dec 29, 2022
C++ Implementation of "An Equivariant Filter for Visual Inertial Odometry", ICRA 2021

EqF VIO (Equivariant Filter for Visual Inertial Odometry) This repository contains an implementation of an Equivariant Filter (EqF) for Visual Inertia

null 60 Nov 15, 2022
R3live - A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package

R3LIVE A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package News [Dec 31, 2021] Release of cod

HKU-Mars-Lab 1.3k Jan 4, 2023
The official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Graph Optimizer This repo contains the official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averagin

Chenyu 109 Dec 23, 2022
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV Workshop @ CVPR 2021.

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022
A simple facial recognition script using OpenCV's FaceRecognizer module implemented in C++

Local Binary Patterns Histogram Recognizer A proyect that implements the LBPHRecognizer class of the OpenCV library to determine if a detected face co

Pablo Agustín Ortega-Kral 0 Jan 18, 2022
Number recognition with MNIST on Raspberry Pi Pico + TensorFlow Lite for Microcontrollers

About Number recognition with MNIST on Raspberry Pi Pico + TensorFlow Lite for Microcontrollers Device Raspberry Pi Pico LCDディスプレイ 2.8"240x320 SPI TFT

iwatake 51 Dec 16, 2022
In this tutorial, we will use machine learning to build a gesture recognition system that runs on a tiny microcontroller, the RP2040.

Pico-Motion-Recognition This Repository has the code used on the 2 parts tutorial TinyML - Motion Recognition Using Raspberry Pi Pico The first part i

Marcelo Rovai 19 Nov 3, 2022
This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicity.

Fast Face Classification (F²C) This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicit

null 33 Jun 27, 2021
WeNet is to close the gap between research and production end-to-end (E2E) speech recognition models,

WeNet is to close the gap between research and production end-to-end (E2E) speech recognition models, to reduce the effort of productionizing E2E models, and to explore better E2E models for production.

null 2.7k Jan 5, 2023