#DeepI2P: Image-to-Point Cloud Registration via Deep Classification
PyTorch implementation for our CVPR 2021 paper DeepI2P. DeepI2P solves the problem of cross modality registration, i.e, solve the relative rotation
R and translation
t between the camera and the lidar.
data: Generate and process datasets
evaluation: Registration codes, include Inverse Camera Projection, ICP, PnP
frustum_reg: C++ codes of the Inverse Camera Projection, using Gauss-Newton Optimization. Installation method is shown below. It requires the Ceres Solver.
python evaluation/frustum_reg/setup.py install
icp: codes for ICP (Iterative Closest Point)
registration_lsq.py: Python code for Inverse Camera Projection, which utilizes the per-point coarse classification prediction, and the
registration_pnp.py: Python code for PnP solver utilizing the per-point fine classification prediction.
kitti: Training codes for KITTI
nuscenes: Training codes for nuscenes
oxford: Training codes for Oxford Robotcar dataset
models: Networks and layers
- 'index_max_ext': This is a custom operation from SO-Net, which is the backbone of our network. Installation:
python models/index_max_ext/setup.py install
networks_img.py: Network to process images. It is a resnet-like structure.
networks_pc.py: Network to process point clouds, it is from SO-Net
network_united.py: Network to fuse information between point clouds and images.