Tightly coupled GNSS-Visual-Inertial system for locally smooth and globally consistent state estimation in complex environment.

Overview

GVINS

GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation. paper link

Authors: Shaozu CAO, Xiuyuan LU and Shaojie SHEN

GVINS is a non-linear optimization based system that tightly fuses GNSS raw measurements with visual and inertial information for real-time and drift-free state estimation. By incorporating GNSS pseudorange and Doppler shift measurements, GVINS is capable to provide smooth and consistent 6-DoF global localization in complex environment. The system framework and VIO part are adapted from VINS-Mono. Our system contains the following features:

  • global 6-DoF estimation in ECEF frame;
  • multi-constellation support (GPS, GLONASS, Galileo, BeiDou);
  • online local-ENU frame alignment;
  • global pose recovery in GNSS-unfriendly or even GNSS-denied area.

1. Prerequisites

1.1 C++11 Compiler

This package requires some features of C++11.

1.2 ROS

This package is developed under ROS Kinetic environment.

1.3 Eigen

Our code uses Eigen 3.3.3 for matrix manipulation.

1.4 Ceres

We use ceres 1.12.0 to solve the non-linear optimization problem.

1.5 gnss_comm

This package also requires gnss_comm for ROS message definitions and some utility functions. Follow those instructions to build the gnss_comm package.

2. Build GVINS

Clone the repository to your catkin workspace (for example ~/catkin_ws/):

cd ~/catkin_ws/src/
git clone https://github.com/HKUST-Aerial-Robotics/GVINS.git

Then build the package with:

cd ~/catkin_ws/
catkin_make
source ~/catkin_ws/devel/setup.bash

If you encounter any problem during the building of GVINS, we recommend you to try docker first.

3. Docker Support

To simplify the building process, we add docker in our code. Docker is like a sandbox so it can isolate our code from your local environment. To run with docker, first make sure ros and docker are installed on your machine. Then add your account to docker group by sudo usermod -aG docker $USER. Logout and re-login to avoid the Permission denied error, then type:

cd ~/catkin_ws/src/GVINS/docker
make build

The docker image gvins:latest should be successfully built after a while. Then you can run GVINS with:

./run.sh LAUNCH_FILE

(for example ./run.sh visensor_f9p.launch). Open another terminal and play your rosbag file, then you should be able to see the result. If you modify the code, simply re-run ./run.sh LAUNCH_FILE to update.

4. Run GVINS with our dataset

Download our GVINS-Dataset and launch GVINS via:

roslaunch gvins visensor_f9p.launch

Open another terminal and launch the rviz by:

rviz -d ~/catkin_ws/src/GVINS/config/gvins_rviz_config.rviz

Then play the bag:

rosbag play sports_field.bag

5. Acknowledgements

The system framework and VIO part are adapted from VINS-Mono. We use camodocal to model the camera and ceres to solve the optimization problem.

6. Licence

The source code is released under GPLv3 license.

Issues
  • Running GVINS on ARM device

    Running GVINS on ARM device

    @shaozu Hi,thanks for your nice work.

    I just started working on GVINS and I'm trying to use this project for real-time drone navigation. That means, I need to run the program on ARM embedded computer.

    Firstly I run GVINS on my PC and laptop and get nice result(sports_field.bag). IMG_7646(20211215-171818)

    Then I run GVINS on NVIDIA Jetson TX2 and NVIDIA Jetson Xavier NX, but I got a problem:

    1. The gnss_enu_path seems to converge slowly and the results don't look as good as those on my PC/laptop;
    2. The path seems to start at a wrong start point and yaw angle.

    As these pictures show (Jetson TX2 with CUDA in feature_tracker): IMG_7635 IMG_7637 IMG_7638 IMG_7641

    And then I use CUDA in feature_tracker to speed up calculations to reduce CPU load, but it didn't seem to help.

    So is this result due to the lack of computing power on embedded computer?? But the CPU load is not too heavy...... Or is it processor architecture (ARM and x86)?

    opened by Mil1ium 12
  •  time-synchronized logic with UTC

    time-synchronized logic with UTC

    Dear Officer,

    Thank you for sharing your great job. I wanna ask about the time-synchronized logic here. https://github.com/HKUST-Aerial-Robotics/GVINS/blob/c3885792c55f996f78807445fa2707f64af51d84/estimator/src/estimator_node.cpp#L274

    Here: time_diff_gnss_visensor = next_pulse_time - trigger_msg->header.stamp.toSec() + LEAP_SECONDS;

    next_pulse_time: pps GPS time in UTC format, remove the leap seconds in epoch2time() already? Correct me if I am wrong trigger_msg->header.stamp.toSec(): VI device's UTC time LEAP_SECONDS : 18seconds

    As the system is time_sync and gps_time = utc_time + LEAP_SECONDS according to here.http://leapsecond.com/java/gpsclock.htm Why do you need to add the LEAP_SECONDS to calculate the time_diff?

    Print the time difference of time_diff_gnss_visensor is quite large. I am running with data2, complex_environment.bag image

    opened by DarrenWong 7
  • gnss_enable is enabled, can not get global path

    gnss_enable is enabled, can not get global path

    @shaozu Hello, disturb to again. I have a question about how to enable gnss_enable.

    as you can see #25. If I set gnss_enable:0, it can get the nice local position. however, when I set gnss_enable:1, it can not get the gloabl position. So i start to debug and get some questions. hope your reply. question: 1: I found that I have no topic /external_trigger is published. how does it work? or is it must? 2: what 's the function of the parameter gnss_local_time_diff ? 3: Is there some reference for explanation gnss_iono_default_parameters. sorry, I have not found any reference. Can it be changed? what it depends on? 4: In GNSSVIAlign() function, I found it always return false as the reason of gnss_meas_buf[i].empty() -> false && gnss_meas_buf[i].size() = 0. https://github.com/HKUST-Aerial-Robotics/GVINS/blob/d2cf40b49c6eb0e6ad3caa4f613713983be3fd74/estimator/src/estimator.cpp#L566-L570 But I can confirm that it has push the data in the buffer here https://github.com/HKUST-Aerial-Robotics/GVINS/blob/d2cf40b49c6eb0e6ad3caa4f613713983be3fd74/estimator/src/estimator.cpp#L326-L327. Very confused me.

    Thank you for sharing your great job.

    opened by Abner0907 4
  • Issues when enabling gnss

    Issues when enabling gnss

    Hi, thanks for the nice work. The demo sports_field really shows perfect performance.

    1. What's the necessary condition to generate the global path?

    The local gvins/path can be generated with RealSense D435i without gnss. But after piping the gnss topics correctly (also using ublox_driver from your repo), the altitude and longitude information do not show up and gvins/enu_path is not generated. The yaml setup for gnss part is the same as provided file.

    2. The following parameters do not show up in the code. What's the function of it?

    gnss_iono_default_parameters: !!opencv-matrix   # unused parameters
       rows: 1
       cols: 8
       dt: d
       data: [0.1118E-07,  0.2235E-07, -0.4172E-06,  0.6557E-06,
              0.1249E+06, -0.4424E+06,  0.1507E+07, -0.2621E+06]
    
    opened by airuchen 4
  • Could not find the required component 'gnss_comm'.

    Could not find the required component 'gnss_comm'.

    system:Ubuntu 18.04.4 eigen:3.3.4 ceres:1.14 (but there is no error in build gnss_comm)

    I have build the gnss_comm package as instruction

    mkdir - p gnss_comm_ws/src
    cd gnss_comm_ws/src 
    git clone https://github.com/HKUST-Aerial-Robotics/gnss_comm.git
    cd ..
    catkin_make
    source ./devel/setup.bash
    

    there is no error then I build GVINS in another workspace as follows

    mkdir -p GVINS_ws/src
    cd GVINS_ws/src
    git clone https://github.com/HKUST-Aerial-Robotics/GVINS.git
    cd ..
    catkin_make
    

    the error is:

    Could not find the required component 'gnss_comm'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
    CMake Error at /opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
      Could not find a package configuration file provided by "gnss_comm" with
      any of the following names:
    
        gnss_commConfig.cmake
        gnss_comm-config.cmake
    
      Add the installation prefix of "gnss_comm" to CMAKE_PREFIX_PATH or set
      "gnss_comm_DIR" to a directory containing one of the above files.  If
      "gnss_comm" provides a separate development package or SDK, be sure it has
      been installed.
    Call Stack (most recent call first):
      GVINS/estimator/CMakeLists.txt:8 (find_package)
    
    

    so if I have to give path in right path? or other method?

    opened by jingshaojing 4
  • Question about UBLOX Config

    Question about UBLOX Config

    hello @shaozu I have follow your steps, do the VI slam, The result is below, looks good. Screenshot from 2021-12-17 16-26-39

    I have two question, hopes your suggestions: question 1: I have a f9p module, but I have not rtcm data, so it will never get rtk fix. Can this data fuse with the image&imu data? question 2: I use my f9p module, run the command, roslaunch ublox_driver ublox_driver.launch, I can see the topic show as below:

    /ublox_driver/ephem /ublox_driver/glo_ephem /ublox_driver/iono_params /ublox_driver/range_meas /ublox_driver/receiver_lla /ublox_driver/receiver_pvt /ublox_driver/time_pulse_info

    How ever, when I rostopic echo /ublox_driver/receiver_lla, no topic is published/display. Do I need some special config through u-center or some other tools?

    opened by Abner0907 3
  • How to dispaly the maps

    How to dispaly the maps

    @shaozu Hello, I am very like this project. it is my needed. I have run it in my pc. all works good. Thank you very much for you nice work.

    I have a simple question: I want know how to display this path trajectory/path in the maps as yours.what tools should i use. how to configure it? I am very appreciate that you can give some reference. thank you very much again. I can display the path in rviz without any map backgrounds. Screenshot from 2021-12-14 15-34-57

    opened by Abner0907 3
  • unit2rcv_pos in code

    unit2rcv_pos in code

    Hello I find you use dopp to constrain the position in code as follow Eigen::Matrix3d unit2rcv_pos; for (size_t i = 0; i < 3; ++i) { for (size_t j = 0; j < 3; ++j) { if (i == j) unit2rcv_pos(i, j) = (norm2-rcv2sat_ecef(i)*rcv2sat_ecef(i))/norm3; else unit2rcv_pos(i, j) = (-rcv2sat_ecef(i)*rcv2sat_ecef(j))/norm3; } } but this method don't mention in your paper, so can you give me some papers about this method

    opened by dean1314 3
  • How can I get time of world frame?

    How can I get time of world frame?

    Hi, thanks for your great work. I want to evaluate accuracy of GVINS(without GNSS) which means I need time of reference frame (or world frame). How can I get this? Thanks in advance.

    opened by Weirdo-p 3
  • IMU noise unit

    IMU noise unit

    I am confused to the units of IMU parameters when I want to run GVINS with my own dataset. Would you mind if you could describe them thoroughly.

    #imu parameters       The more accurate parameters you provide, the better performance
    acc_n: 0.08          # accelerometer measurement noise standard deviation. #0.2   0.04
    gyr_n: 0.004         # gyroscope measurement noise standard deviation.     #0.05  0.004
    acc_w: 0.00004         # accelerometer bias random work noise standard deviation.  #0.02
    gyr_w: 2.0e-6       # gyroscope bias random work noise standard deviation.     #4.0e-5
    
    opened by Weirdo-p 3
  • Stereo Compatibility

    Stereo Compatibility

    Hello and thank you for a great piece of work! From what I gather, there is nothing stopping this method working with a pair of stereo cameras (such as the RS D455); I was just wondering what the changes needed to be done in the codebase would be and if you are looking forward to doing it

    opened by ardabbour 3
  • About the Earth's rotation

    About the Earth's rotation

    The GVINS paper and code are very clear and helpful. I have a small question about the code.

    When reading 'gnss_psr_dopp_factor.cpp', I cannot find code segment that corresponds to the Earth's rotation term of formulas (16) and (20) of the GVINS paper. Is this part of the processing omitted from the code? And have you tested how much the Earth's rotation affects accuracy?

    Looking forward to your reply!

    opened by yinzhang809 0
  • Migrate to VINS-Fusion framework

    Migrate to VINS-Fusion framework

    Hi @shaozu

    Thanks for your awesome work.

    Right now, I was trying to migrate from VINS-Mono to VINS-Fusion framework in order to get stereo VIO support, and I encountered a weird problem that the ECEF anchor point didn't converge.

    image

    As the above figure shown, the green path is the RTK groundtruth, and the purple one is the VIO odometry path, while the red path is the fused path. We can see that the fused position jittered a lot because of the unstable ECEF anchor position.

    If I set the ECEF anchor point as constant, the fused path will be smooth, but the accuracy is almost the same as VIO.

    So I was wondering if you ever encounter this issue before.

    Looking forward to your response.

    Thanks.

    opened by sunbingfeng 1
  • Issuses on the Jacobian of the GnssPsrDoppFactor.

    Issuses on the Jacobian of the GnssPsrDoppFactor.

    Dear authors: Could you please tell me how to calculate the doppler factor Jacobian with respect to the position state? I do not understand the codes that are marked by a red cycle in the following figure. I refer to the arxiv version of your article and do not find the details about this Jacobian. Could you give me a guide or reference? Thank you! 多普勒因子对位置的雅克比

    opened by Nronaldo 2
  • How to record data for GVINs with phone?

    How to record data for GVINs with phone?

    @shaojie
    Hello, Your work is pretty impressed, it works perfect on pc. Next step I'd like to run your algorithm on android, however, it seems hard to get same data as you provided for GVINS. The Gnss Measurement offerd by Google is subset of what GVINs need. Have you teams tried to run GVINS on phone? Do you have tools for collect GNSS data needed by GVINs with phone? Thanks in advance!

    opened by Chanchanhan 0
  • Gnss does not initialize

    Gnss does not initialize

    Hi, I have collected a dataset where the "/ublox_driver/range_meas" publish at 1 Hz, from your published dataset I see that the same topic publishes at 10 Hz. The visual-inertial system initializes well for mye dataset, but the GNSS estimate does not. Is there a minimum requirement for the frequency of the "/ublox_driver/range_meas" topic for the gnss estimate to initialize? The GNSSVIAlign() function seems to be stopped by the following line:

    for (uint32_t i = 0; i < (WINDOW_SIZE+1); ++i) { if (gnss_meas_buf[i].empty() || gnss_meas_buf[i].size() < 10){ return false; } } So I guess my question is if this seems to be a data-set issue or something that can be fixed by changing some thresholds in the code?

    opened by maghauke 1
Owner
HKUST Aerial Robotics Group
HKUST Aerial Robotics Group
RRxIO - Robust Radar Visual/Thermal Inertial Odometry: Robust and accurate state estimation even in challenging visual conditions.

RRxIO - Robust Radar Visual/Thermal Inertial Odometry RRxIO offers robust and accurate state estimation even in challenging visual conditions. RRxIO c

Christopher Doer 56 Jul 26, 2022
A real-time, direct and tightly-coupled LiDAR-Inertial SLAM for high velocities with spinning LiDARs

LIMO-Velo [Alpha] ?? [16 February 2022] ?? The project is on alpha stage, so be sure to open Issues and Discussions and give all the feedback you can!

Andreu Huguet 121 Aug 12, 2022
Iot-Surveillance-Car - This is a IOT Based Surveillance Car which can be controlled, tracked globally as well as its data can be accessed globally

Iot-Surveillance-Car - This is a IOT Based Surveillance Car which can be controlled, tracked globally as well as its data can be accessed globally. The camera on the front of the car can also be monitored globally. It can go anywhere where sim connection is available. 5th Sem Mini project

Rahul Vijan 3 Jun 29, 2022
A dataset containing synchronized visual, inertial and GNSS raw measurements.

GVINS-Dataset Author/Maintainer: CAO Shaozu (shaozu.cao AT gmail.com), LU Xiuyuan (xluaj AT connect.ust.hk) This repository hosts dataset collected du

HKUST Aerial Robotics Group 104 Jul 25, 2022
VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation

VID-Fusion VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation Authors: Ziming Ding , Tiankai Yang, Kunyi Zhan

ZJU FAST Lab 77 Aug 2, 2022
ROS GNSS/INS driver for Inertial Labs positioning systems for the CARMA Platform

CARMA Inertial Labs GNSS/INS Driver This is a fork of the Inertial Labs ROS package that is used for connecting to Inertial Labs GNSS/INS, IMU-P, AHRS

null 2 Dec 26, 2021
The movements of your RC vehicles are jerky and not smooth? This Arduino device will solve this issue by adding acceleration and deceleration ramps to the PWM signals!

This is an Arduino Pro Mini 3.3V / 8MHz based RC servo ramp / delay generator Features: 4 RC servo PWM inputs and outputs (can be enhanced) Reads the

null 4 Apr 15, 2022
Non-blocking blinking patterns and smooth fade effects for your LEDs and buzzers.

Arduino-Blinkenlight Featured on hackster.io! ?? Supercharge your status-LEDs ?? This library gives you non-blocking blinking patterns and smooth fade

Thomas Feldmann 19 Aug 4, 2022
Terrain generator with 5 visually distinct biomes, spread them in regions and smooth their borders

termProject This is the repository of my term project: a terrain generator. Abstract The larger the scenario of a game, more time a player will spend

João Carlos Becker 13 Jun 9, 2021
Best Method to get Globally Banned from the cfx.re community

Lua-Executor Best Method to get Globally Banned from the cfx.re community Since cheaters have been going crazy selling 'their' hacks, and often gets d

Scopes 7 Jul 2, 2022
Wayfire plugin for handling touchpad gestures globally in a layer-shell surface

wf-globalgestures Global touchpad gestures plugin for Wayfire: implements a special protocol (also in this repo) that lets clients request that a part

null 4 Jan 18, 2022
Continuous-Time Spline Visual-Inertial Odometry

Continuous-Time Spline Visual-Inertial Odometry Related Publications Direct Sparse Odometry, J. Engel, V. Koltun, D. Cremers, In IEEE Transactions on

Minnesota Interactive Robotics and Vision Laboratory 46 Aug 9, 2022
Visual-inertial-wheel fusion odometry, better performance in scenes with drastic changes in light

VIW-Fusion An visual-inertial-wheel fusion odometry VIW-Fusion is an optimization-based viusla-inertial-wheel fusion odometry, which is developed as a

庄庭达 189 Aug 6, 2022
RR4J is a tool that records java execution and later allows developers to replay locally.

RR4J [Record Replay 4 Java] RR4J is a tool that records java execution and later allows developers to replay locally. The tool solves one of the chall

Kartik  kalaghatgi 15 Aug 7, 2022
C++ Library to Execute Leetcode Problems Locally

C++ Library to Execute Leetcode Problems Locally This repository contains a C++ library that helps to execute Leetcode problems in one line. header-on

Yuheng HUANG 7 Apr 7, 2022
Yet another matrix client. Click packaging for locally running on Ubuntu Touch

Cinny Click Packaging Cinny is a Matrix client focusing primarily on simple, elegant and secure interface. License Cinny source package licensed under

Nitan Alexandru Marcel 5 Jun 24, 2022
Optimization-Based GNSS/INS Integrated Navigation System

OB_GINS Optimization-Based GNSS/INS Integrated Navigation System We open-source OB_GINS, an optimization-based GNSS/INS integrated navigation system.

i2Nav-WHU 211 Aug 8, 2022
Visual Leak Detector for Visual C++ 2008-2015

Visual Leak Detector Introduction Visual C++ provides built-in memory leak detection, but its capabilities are minimal at best. This memory leak detec

Arkady Shapkin 870 Aug 5, 2022
fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.

fx is a workspace tool manager. It allows you to create consistent, discoverable, language-neutral and developer friendly command line tools.

null 18 Jun 2, 2022