RP-VIO: Robust Plane-based Visual-Inertial Odometry for Dynamic Environments (Code & Dataset)

Hi, After install the third_party. I get this error:

2021-06-28 22:43:38.525918: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Unsuccessful TensorSliceReader constructor: Failed to get matching files on pre_trained_model/synthia_498000: Not found: pre_trained_model

My computer is Laptop Thinkpad P17, and the GPU is the RTX3000 Is this error with the GPU ?

I used the ol_market1_config.yaml file of your rp-vio project to directly run VINS-MONO on the OpenLORIS-market-1 dataset, and got a higher accuracy than your paper (paper is 2.45, but I got Absolute Trajectory RMSE = 0.88). Why?

The evaluation tool I use is EVO, and the link is [https://github.com/MichaelGrupp/evo]

The command is evo_ape tum groundtruth.txt vins_result_no_loop.txt -va -p --plot_mode=xyz --align --correct_scale

Looking forward to your answer

Hi, when I run python inference.py, I got this:

Traceback (most recent call last):
  File "inference.py", line 14, in <module>
    from RecoverPlane import RecoverPlane
ModuleNotFoundError: No module named 'RecoverPlane'

How can I fix this error ?

Sorry to bother you again, rviz does not show the trajectory when I run the ADVIO dataset downloaded from your official website, what could be the reason?

Hi, sorry to bother. I try to run the city_night in the code, And I use the config from the raw git https://github.com/kminoda/VIODE/blob/master/calibration_files/cam0_pinhole.yaml But The performance is bad, do you know I did any thing wrong? Thanks Here is my config

%YAML:1.0

#common parameters imu_topic: "/imu0" image_topic: "/cam0/image_raw" mask_topic: "/cam0/segmentation" output_path: "/home/zhouxin/output/"

#camera calibration model_type: PINHOLE camera_name: camera image_width: 752 image_height: 480 distortion_parameters: k1: 0 k2: 0 p1: 0 p2: 0 projection_parameters: fx: 376.0 fy: 376.0 cx: 376.0 cy: 240.0

estimate_extrinsic: 0 #Rotation from camera frame to imu frame, imu^R_cam extrinsicRotation: !!opencv-matrix rows: 3 cols: 3 dt: d data: [1,0,0,0,1,0,0,0,1]

#Translation from camera frame to imu frame, imu^T_cam extrinsicTranslation: !!opencv-matrix rows: 3 cols: 1 dt: d data: [0,0,0]

#feature traker paprameters max_cnt: 120 # max feature number in feature tracking min_dist: 30 # min distance between two features freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image F_threshold: 1.0 # ransac threshold (pixel) show_track: 1 # publish tracking image as topic flow_back: 1 # perform forward and backward optical flow to improve feature tracking accuracy

#optimization parameters max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time max_num_iterations: 8 # max solver itrations, to guarantee real time keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters The more accurate parameters you provide, the better performance acc_n: 0.2 # accelerometer measurement noise standard deviation. gyr_n: 0.05 # gyroscope measurement noise standard deviation.
acc_w: 0.02 # accelerometer bias random work noise standard deviation.
gyr_w: 4.0e-5 # gyroscope bias random work noise standard deviation.
g_norm: 9.81007 # gravity magnitude

#unsynchronization parameters estimate_td: 0
td: 0.0 # initial value of time offset. unit s. readed image clock + td = real image clock (IMU clock)

#rolling shutter parameters rolling_shutter: 0 # 0: global shutter camera, 1: rolling shutter camera rolling_shutter_tr: 0 # unit: s. rolling shutter read out time per frame (from data sheet).

#visualization parameters save_image: 1 # save image in pose graph for visualization prupose; you can close this function by setting 0 visualize_imu_forward: 0 # output imu forward propogation to achieve low latency and high frequence results visualize_camera_size: 0.4 # size of camera marker in RVIZ

Hi，

I appreciate you can answer my question,

Thank you for your open source code and datasets.

Your work has been of a great help to my research.

I'd like to ask you another question.

For the market1-1 sequence from the OpenLORIS-Scene dataset, how do you align the data of d435i accelerometer and gyroscope to one topic /imu0?

And Could you open source the datasets market1-2 and market1-3 that align imu0 data? I also want to test it.

Thank you very much!

Hello, thanks for sharing. I run the datasets 12-mask.bag and have this error: $roslaunch rpvio_estimator advio_12.launch bagfile_path:=/media/li/DATA/DATASET/RP-VIO/ADVIO/12-mask.bag OpenCV Error: Unknown error code -28 (The input arrays should have at least 4 corresponding point sets to calculate Homography) in findHomography Did you ever meet this situation? By the way, I have tried other datasets and also met this problem. The modifications I made to the code were only adding 'std::' in the 'map<int, std::array<double,3>> para_N;' because 'error: template argument 2 is invalid #14' as mentioned.

Hi, Sorry to bothering you. But when I install some package that needed for plane recover library. There are also show:

/home/robotics/.local/lib/python3.6/site-packages/scipy/__init__.py:147: UserWarning: NumPy 1.14.5 or above is required for this version of SciPy (detected version 1.13.1)
  UserWarning)
Traceback (most recent call last):
  File "inference.py", line 14, in <module>
    from RecoverPlane import RecoverPlane
ModuleNotFoundError: No module named 'RecoverPlane'

But I found the RecoverPlane is in RecoverPlane_perpendicular.py python file, but I don't know how to let the inference.py to find this class ?

Hi，I'm a freshman to the groundplane segmentation algorithm. About the inference part, how to edit the test list in the command, Like in readme.md:

 --test_list=<TEST_DATA_LIST.txt FILE> 

for example, if I want to segment the ground plane in the left image in kitti, what should I do？ could you please give me an example about it？ I'm appreciate a lot for your wonderful help.

Hi thanks for sharing your work. I want to run the RP-VIO in real time on pixhawk drone. How can provide mask(segmented) plane of camera? I am using mynt-eye and realsense 435i camera thanks

Hi, I run vins-mono with your config file 3 times and got 1.55, 2.22, and 1.65 rmse. The results from VINS-Mono are more than RP-VIO.

Have you run the segmentation part ? When I run python inference.py, I got this:

from RecoverPlane import RecoverPlane ModuleNotFoundError: No module named 'RecoverPlane

Have you met this error before?

Originally posted by @Gatsby23 in https://github.com/karnikram/rp-vio/issues/2#issuecomment-867651960

Hi, Thanks for sharing your remarkable work! I want to run the RP-VIO algorithm on KAIST urban dataset, which is a highway scene, and i segmented road only. However, the code throws warnings "No enough features or parallex; Move device around" all the time as following screenshot, and the output of the system is an empty "*.csv" file. Could you give me some advices to fix such problems? Looking forward to your reply!

Hi, I try to segment all the sequences in ADVIO dataset. I successfully segment some cases like 10, 13, 23 and the rmse is great. But I notice that the segmentation model is not so workable in highlighted corrior like in case 17. And I also find that the distortion influences the results badly. In case 12, the segmentation results is better than the reults that I dont use the dirtortion coeffs. So do you have any good solutions to the two problems, like some better distortion coeffs or models. Thanks a lot！ And here is my results. https://1drv.ms/u/s!AgIBb31yefjshg6ObUe0bhNB6OJR?e=Ej8Vol

Hi, I try to use the plane segmentation in the ADVIO dataset you provides, however it cant perforn like the mask you provide after CRF interface. I use the pretrained model you provide. and the raw image has resized already. But this stiil some hard code in interface.py like plane_num. I dont know what I did wrong, could pls help me?