@shaozu Hi,thanks for your nice work.

I just started working on GVINS and I'm trying to use this project for real-time drone navigation. That means, I need to run the program on ARM embedded computer.

Firstly I run GVINS on my PC and laptop and get nice result(sports_field.bag).

Then I run GVINS on NVIDIA Jetson TX2 and NVIDIA Jetson Xavier NX, but I got a problem:

1. The gnss_enu_path seems to converge slowly and the results don't look as good as those on my PC/laptop;
2. The path seems to start at a wrong start point and yaw angle.

As these pictures show (Jetson TX2 with CUDA in feature_tracker):

And then I use CUDA in feature_tracker to speed up calculations to reduce CPU load, but it didn't seem to help.

So is this result due to the lack of computing power on embedded computer?? But the CPU load is not too heavy...... Or is it processor architecture (ARM and x86)?

Dear Officer,

Thank you for sharing your great job. I wanna ask about the time-synchronized logic here. https://github.com/HKUST-Aerial-Robotics/GVINS/blob/c3885792c55f996f78807445fa2707f64af51d84/estimator/src/estimator_node.cpp#L274

Here: time_diff_gnss_visensor = next_pulse_time - trigger_msg->header.stamp.toSec() + LEAP_SECONDS;

next_pulse_time: pps GPS time in UTC format， remove the leap seconds in epoch2time() already? Correct me if I am wrong trigger_msg->header.stamp.toSec(): VI device's UTC time LEAP_SECONDS ： 18seconds

As the system is time_sync and gps_time = utc_time + LEAP_SECONDS according to here.http://leapsecond.com/java/gpsclock.htm Why do you need to add the LEAP_SECONDS to calculate the time_diff?

Print the time difference of time_diff_gnss_visensor is quite large. I am running with data2, complex_environment.bag

@shaozu Hello, disturb to again. I have a question about how to enable gnss_enable.

as you can see #25. If I set gnss_enable:0, it can get the nice local position. however, when I set gnss_enable:1, it can not get the gloabl position. So i start to debug and get some questions. hope your reply. question: 1: I found that I have no topic /external_trigger is published. how does it work? or is it must? 2: what 's the function of the parameter gnss_local_time_diff ? 3: Is there some reference for explanation gnss_iono_default_parameters. sorry, I have not found any reference. Can it be changed? what it depends on? 4: In GNSSVIAlign() function, I found it always return false as the reason of gnss_meas_buf[i].empty() -> false && gnss_meas_buf[i].size() = 0. https://github.com/HKUST-Aerial-Robotics/GVINS/blob/d2cf40b49c6eb0e6ad3caa4f613713983be3fd74/estimator/src/estimator.cpp#L566-L570 But I can confirm that it has push the data in the buffer here https://github.com/HKUST-Aerial-Robotics/GVINS/blob/d2cf40b49c6eb0e6ad3caa4f613713983be3fd74/estimator/src/estimator.cpp#L326-L327. Very confused me.

Thank you for sharing your great job.

Hi, thanks for the nice work. The demo sports_field really shows perfect performance.

1. What's the necessary condition to generate the global path?

The local gvins/path can be generated with RealSense D435i without gnss. But after piping the gnss topics correctly (also using ublox_driver from your repo), the altitude and longitude information do not show up and gvins/enu_path is not generated. The yaml setup for gnss part is the same as provided file.

2. The following parameters do not show up in the code. What's the function of it?

gnss_iono_default_parameters: !!opencv-matrix   # unused parameters
   rows: 1
   cols: 8
   dt: d
   data: [0.1118E-07,  0.2235E-07, -0.4172E-06,  0.6557E-06,
          0.1249E+06, -0.4424E+06,  0.1507E+07, -0.2621E+06]

system:Ubuntu 18.04.4 eigen:3.3.4 ceres:1.14 (but there is no error in build gnss_comm)

I have build the gnss_comm package as instruction

mkdir - p gnss_comm_ws/src
cd gnss_comm_ws/src 
git clone https://github.com/HKUST-Aerial-Robotics/gnss_comm.git
cd ..
catkin_make
source ./devel/setup.bash

there is no error then I build GVINS in another workspace as follows

mkdir -p GVINS_ws/src
cd GVINS_ws/src
git clone https://github.com/HKUST-Aerial-Robotics/GVINS.git
cd ..
catkin_make

the error is:

Could not find the required component 'gnss_comm'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
CMake Error at /opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
  Could not find a package configuration file provided by "gnss_comm" with
  any of the following names:

    gnss_commConfig.cmake
    gnss_comm-config.cmake

  Add the installation prefix of "gnss_comm" to CMAKE_PREFIX_PATH or set
  "gnss_comm_DIR" to a directory containing one of the above files.  If
  "gnss_comm" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  GVINS/estimator/CMakeLists.txt:8 (find_package)

so if I have to give path in right path? or other method?

hello @shaozu I have follow your steps, do the VI slam, The result is below, looks good.

I have two question, hopes your suggestions: question 1: I have a f9p module, but I have not rtcm data, so it will never get rtk fix. Can this data fuse with the image&imu data? question 2: I use my f9p module, run the command, roslaunch ublox_driver ublox_driver.launch, I can see the topic show as below:

/ublox_driver/ephem /ublox_driver/glo_ephem /ublox_driver/iono_params /ublox_driver/range_meas /ublox_driver/receiver_lla /ublox_driver/receiver_pvt /ublox_driver/time_pulse_info

How ever, when I rostopic echo /ublox_driver/receiver_lla, no topic is published/display. Do I need some special config through u-center or some other tools?

@shaozu Hello, I am very like this project. it is my needed. I have run it in my pc. all works good. Thank you very much for you nice work.

I have a simple question: I want know how to display this path trajectory/path in the maps as yours.what tools should i use. how to configure it? I am very appreciate that you can give some reference. thank you very much again. I can display the path in rviz without any map backgrounds.

Hello I find you use dopp to constrain the position in code as follow Eigen::Matrix3d unit2rcv_pos; for (size_t i = 0; i < 3; ++i) { for (size_t j = 0; j < 3; ++j) { if (i == j) unit2rcv_pos(i, j) = (norm2-rcv2sat_ecef(i)*rcv2sat_ecef(i))/norm3; else unit2rcv_pos(i, j) = (-rcv2sat_ecef(i)*rcv2sat_ecef(j))/norm3; } } but this method don't mention in your paper, so can you give me some papers about this method

Hi, thanks for your great work. I want to evaluate accuracy of GVINS(without GNSS) which means I need time of reference frame (or world frame). How can I get this? Thanks in advance.

I am confused to the units of IMU parameters when I want to run GVINS with my own dataset. Would you mind if you could describe them thoroughly.

#imu parameters       The more accurate parameters you provide, the better performance
acc_n: 0.08          # accelerometer measurement noise standard deviation. #0.2   0.04
gyr_n: 0.004         # gyroscope measurement noise standard deviation.     #0.05  0.004
acc_w: 0.00004         # accelerometer bias random work noise standard deviation.  #0.02
gyr_w: 2.0e-6       # gyroscope bias random work noise standard deviation.     #4.0e-5

Hello and thank you for a great piece of work! From what I gather, there is nothing stopping this method working with a pair of stereo cameras (such as the RS D455); I was just wondering what the changes needed to be done in the codebase would be and if you are looking forward to doing it

Hey all! I have been trying out GVINS on my handheld devices (ZED2i stereo - rolling shutter, OAK-D - global shutter).

I keep getting the following error: [ INFO] [1670628907.037916833]: Not enough features or parallax; Move device around

I have tried with both devices and i get the following error when running in pure VI mode. I initially thought it could be an issue with the camera+IMU calibration. But with the same parameters, I did the calibration using the kalibr package and was able to run ORB SLAM3 and VINS-Fusion (stereo + IMU) for both cameras. So, I am unsure about what could be the issue.

The config files have been set properly. I am running GVINS using just one of the cameras in both ZED and OAK-D (since it's mono mode).

The only difference I see in the config file is the declaration of extrinsic parameters. In VINS-Fusion - for my ZED camera

# Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 0   # 0  Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
                        # 1  Have an initial guess about extrinsic parameters. We will optimize around your initial guess.

body_T_cam0: !!opencv-matrix
   rows: 4
   cols: 4
   dt: d
   data:  [ 0.0051989,  -0.00475195,  0.99997519,  0.01782353,
           -0.99998009, -0.00360067,  0.00518181,  0.02405353,
            0.00357595, -0.99998223, -0.00477058,  0.00652621,
            0.,          0.,          0.,          1.        ]

body_T_cam1: !!opencv-matrix
   rows: 4
   cols: 4
   dt: d
   data: [0.01907386, -0.00099337, 0.99981758, 0.01775441,
           -0.9998151, -0.00245767, 0.01907137, -0.097972 ,
          0.00243827, -0.99999649, -0.00104007, 0.00694309,
          0, 0, 0, 1]

#Multiple thread support
multiple_thread: 1

#feature traker paprameters
max_cnt: 150            # max feature number in feature tracking
min_dist: 30            # min distance between two features 
freq: 0              # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image 
F_threshold: 1.0        # ransac threshold (pixel)
show_track: 1           # publish tracking image as topic
flow_back: 1            # perform forward and backward optical flow to improve feature tracking accuracy

#optimization parameters
max_solver_time: 0.04  # max solver itration time (ms), to guarantee real time
max_num_iterations: 8   # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters       The more accurate parameters you provide, the better performance
acc_n: 0.0015999999595806003       # accelerometer measurement noise standard deviation. 
gyr_n: 0.007000000216066837         # gyroscope measurement noise standard deviation.     
acc_w: 0.0002508999896235764      # accelerometer bias random work noise standard deviation.  
gyr_w: 0.0019474000437185168       # gyroscope bias random work noise standard deviation.     
g_norm: 9.81007     # gravity magnitude

#unsynchronization parameters
estimate_td: 0                      # online estimate time offset between camera and imu
td: 0.0         

while in GVINS it is

# Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 1   # 0  Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
                        # 1  Have an initial guess about extrinsic parameters. We will optimize around your initial guess.
                        # 2  Don't know anything about extrinsic parameters. You don't need to give R,T. We will try to calibrate it. Do some rotation movement at beginning.                        
#If you choose 0 or 1, you should write down the following matrix.
#Rotation from camera frame to imu frame, imu^R_cam
extrinsicRotation: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data:   [0.0051989,  -0.00475195,  0.99997519,
         -0.99998009, -0.00360067,  0.00518181, 
         0.00357595, -0.99998223, -0.00477058]
   
#Translation from camera frame to imu frame, imu^T_cam
extrinsicTranslation: !!opencv-matrix
   rows: 3
   cols: 1
   dt: d
   data: [0.01782353, 0.02405353, 0.00652621]

#feature traker paprameters
max_cnt: 150           # max feature number in feature tracking
min_dist: 30           # min distance between two features 
freq: 10                # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image 
F_threshold: 1.0        # ransac threshold (pixel)
show_track: 1           # publish tracking image as topic
equalize: 1             # if image is too dark or light, trun on equalize to find enough features
fisheye: 0              # if using fisheye, trun on it. A circle mask will be loaded to remove edge noisy points

#optimization parameters
max_solver_time: 0.04  # max solver itration time (ms), to guarantee real time
max_num_iterations: 8   # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters       The more accurate parameters you provide, the better performance
acc_n: 0.0015999999595806003       # accelerometer measurement noise standard deviation. 
gyr_n: 0.007000000216066837         # gyroscope measurement noise standard deviation.     
acc_w: 0.0002508999896235764      # accelerometer bias random work noise standard deviation.  
gyr_w: 0.0019474000437185168       # gyroscope bias random work noise standard deviation.     
g_norm: 9.81007     # gravity magnitude

#unsynchronization parameters
estimate_td: 0                      # online estimate time offset between camera and imu
td: 0.0                             # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)`

Could you help figure out what could be going wrong? I am using the distortion parameters of camera 0 in GVINS.

Any ideas? @shaojie @dvorak0 @WayneTimer @groundmelon @hlx1996

Thanks!!

Thanks for your nice work. @shaozu

I've got a new question about GNSS receiver dt and ddt factor.

In your code, standard deviation of ddt is set to 0.1(unit: m/s ?) and the infomation matrix of dt is set to 50(unit: m ?). Could you please tell me why set them to these values?

Hi~bro. I ran the GVINS with my own device (including a realsense D455 as the VIS and a H-RTK-F9P as the GNSS module ). I set all the device right. And the GPS runs fine. But it shows "length error" in the Ublox-driver terminal windiow.But when I run the GVINS project with the estimate_extrinsic 2. But After I roslaunched the GVINS , the terminal windows shows only "init begins". And I opened the rviz , it had the original image and the feature point image but no pose. I echoed the enu_pose and other messages outputed by GVINS, they are all none. So why is this?

Could you give me some advice about this? Thanks a lot.

Great work, code and paper!

When looking at the datasheet of the ADIS16448 and your paper, your dataset should have X to the right of the helmet, Z axis to the front and Y axis down. This can be confirmed with the bag values (please correct me if I'm wrong)

Similar, the camera matrix rotation need to be corrected to fit the IMU as this is the body frame (as indicated in the paper).

Question: Where do you correct the IMU to real world coordinates system? I have not been able to found it. My coordinate system is not the same (X to the front, Y to the left and Z to the top), what make the algorithm fail

Thanks!

Hi~ I used realsense D455 as the VIsensor and an H-RTK-F9P as the GNSS module. I set the estimate_extrinsic 2 and modified all the topic name But when I run the launch file. It showed this: "init begins" and

why is this? could you offer some advice? Thanks a lot.