This repository contains Python and C++ implementation of Robust Consistent Video Depth, as described in the paper

Related tags

Artificial Intelligence robust_cvd
Overview

[CVPR 2021] Robust Consistent Video Depth Estimation

Open in Colab

This repository contains Python and C++ implementation of Robust Consistent Video Depth, as described in the paper

Johannes Kopf, Xuejian Rong, and Jia-Bin Huang. Robust Consistent Video Despth Estimation. CVPR 2021

Project | Paper | Video | Colab

We present an algorithm for estimating consistent dense depth maps and camera poses from a monocular video. We integrate a learning-based depth prior, in the form of a convolutional neural network trained for single-image depth estimation, with geometric optimization, to estimate a smooth camera trajectory as well as detailed and stable depth reconstruction.

teaser

Changelog

  • [June 2021] Released the companion Colab notebook.
  • [June 2021] Initial release of Robust CVD.

Installation

Please refer to the colab notebook for how to install the dependencies.

Running

Please refer to the colab notebook for how to run the cli tool for now.

Result Folder Structure

frames.txt              # meta data about number of frames, image resolution and timestamps for each frame
color_full/             # extracted frames in the original resolution
color_down/             # extracted frames in the resolution for disparity estimation 
color_down_png/      
color_flow/             # extracted frames in the resolution for flow estimation
flow_list.json          # indices of frame pairs to finetune the model with
flow/                   # optical flow 
mask/                   # mask of consistent flow estimation between frame pairs.
vis_flow/               # optical flow visualization. Green regions contain inconsistent flow. 
vis_flow_warped/        # visualzing flow accuracy by warping one frame to another using the estimated flow. e.g., frame_000000_000032_warped.png warps frame_000032 to frame_000000.
depth_${model_type}/    # initial disparity estimation using the original monocular depth model before test-time training
R_hierarchical2_${model_type}/ 
    flow_list_0.20.json                 # indices of frame pairs passing overlap ratio test of threshold 0.2. Same content as ../flow_list.json.
    videos/                             # video visualization of results 
    B0.1_R1.0_PL1-0_LR0.0004_BS4_Oadam/
        checkpoints/                    # checkpoint after each epoch
        depth/                          # final disparity map results after finishing test-time training
        eval/                           # intermediate losses and disparity maps after each epoch 
        tensorboard/                    # tensorboard log for the test-time training process

Citation

If you find our work useful in your research, please consider citing:

@inproceedings{kopf2021rcvd,
 title={Robust Consistent Video Depth Estimation},
 author={Kopf, Johannes and Rong, Xuejian and Huang, Jia-Bin},
 year={2021},
 booktitle=IEEE/CVF Conference on Computer Vision and Pattern Recognition
}

License

See the LICENSE for more details.

Issues & Help

For help or issues using Robust CVD, please submit a GitHub issue or a PR request.

Before you do this, make sure you have checked CODE_OF_CONDUCT, CONTRIBUTING, ISSUE_TEMPLATE, and PR_TEMPLATE.

Acknowledgements

Check our previous work on Consistent Video Depth Estimation.

We also thank the authors for releasing PyTorch, Ceres Solver, OpenCV, Eigen, MiDaS, RAFT, and detectron2.

Comments
  • while building make on Ubuntu 22.04 I am getting this error. can anyone please help me with this?

    while building make on Ubuntu 22.04 I am getting this error. can anyone please help me with this?

    /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:163:1: error: reference to ‘Vector3’ is ambiguous 163 | Vector3 obsToCamera( | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:175:1: error: reference to ‘Vector3’ is ambiguous 175 | Vector3 cameraToWorld( | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:196:1: error: reference to ‘Vector3’ is ambiguous 196 | Vector3 worldToCamera( | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp: In member function ‘bool facebook::cp::{anonymous}::SceneFlowSmoothnessLoss::operator()(const T* const*, T*) const’: /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:338:5: error: reference to ‘Vector3’ is ambiguous 338 | Vector3 pointCam0 = obsToCamera(obs0, params0); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:338:14: error: expected primary-expression before ‘>’ token 338 | Vector3 pointCam0 = obsToCamera(obs0, params0); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:338:16: error: ‘pointCam0’ was not declared in this scope 338 | Vector3 pointCam0 = obsToCamera(obs0, params0); | ^~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:341:5: error: reference to ‘Vector3’ is ambiguous 341 | Vector3 pointCam1 = obsToCamera(obs1, params1); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:341:14: error: expected primary-expression before ‘>’ token 341 | Vector3 pointCam1 = obsToCamera(obs1, params1); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:341:16: error: ‘pointCam1’ was not declared in this scope 341 | Vector3 pointCam1 = obsToCamera(obs1, params1); | ^~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:344:5: error: reference to ‘Vector3’ is ambiguous 344 | Vector3 pointCam2 = obsToCamera(obs2, params2); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:344:14: error: expected primary-expression before ‘>’ token 344 | Vector3 pointCam2 = obsToCamera(obs2, params2); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:344:16: error: ‘pointCam2’ was not declared in this scope 344 | Vector3 pointCam2 = obsToCamera(obs2, params2); | ^~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:347:5: error: reference to ‘Vector2’ is ambiguous 347 | Vector2 focal0, focal1, focal2; | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector2 = Eigen::Matrix<Type, 2, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:540:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 540 | EIGEN_MAKE_TYPEDEFS(2, 2) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:31:7: note: ‘template using Vector2 = Eigen::Matrix<Type, 2, 1>’ 31 | using Vector2 = Matrix<T, 2, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:347:14: error: expected primary-expression before ‘>’ token 347 | Vector2 focal0, focal1, focal2; | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:347:16: error: ‘focal0’ was not declared in this scope 347 | Vector2 focal0, focal1, focal2; | ^~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:347:24: error: ‘focal1’ was not declared in this scope 347 | Vector2 focal0, focal1, focal2; | ^~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:347:32: error: ‘focal2’ was not declared in this scope 347 | Vector2 focal0, focal1, focal2; | ^~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:362:18: error: template argument 1 is invalid 362 | Map<Vector3> smoothLoss(residuals); | ^~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:369:7: error: reference to ‘Vector3’ is ambiguous 369 | Vector3 pointWorld0 = cameraToWorld(pointCam0, focal0, params0.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:369:16: error: expected primary-expression before ‘>’ token 369 | Vector3 pointWorld0 = cameraToWorld(pointCam0, focal0, params0.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:369:18: error: ‘pointWorld0’ was not declared in this scope 369 | Vector3 pointWorld0 = cameraToWorld(pointCam0, focal0, params0.pose); | ^~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:370:7: error: reference to ‘Vector3’ is ambiguous 370 | Vector3 pointWorld1 = cameraToWorld(pointCam1, focal1, params1.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:370:16: error: expected primary-expression before ‘>’ token 370 | Vector3 pointWorld1 = cameraToWorld(pointCam1, focal1, params1.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:370:18: error: ‘pointWorld1’ was not declared in this scope 370 | Vector3 pointWorld1 = cameraToWorld(pointCam1, focal1, params1.pose); | ^~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:371:7: error: reference to ‘Vector3’ is ambiguous 371 | Vector3 pointWorld2 = cameraToWorld(pointCam2, focal2, params2.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:371:16: error: expected primary-expression before ‘>’ token 371 | Vector3 pointWorld2 = cameraToWorld(pointCam2, focal2, params2.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:371:18: error: ‘pointWorld2’ was not declared in this scope 371 | Vector3 pointWorld2 = cameraToWorld(pointCam2, focal2, params2.pose); | ^~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:375:7: error: reference to ‘Vector3’ is ambiguous 375 | Vector3 pointWorld0 = cameraToWorld(pointCam0, focal0, params0.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:375:16: error: expected primary-expression before ‘>’ token 375 | Vector3 pointWorld0 = cameraToWorld(pointCam0, focal0, params0.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:375:18: error: ‘pointWorld0’ was not declared in this scope 375 | Vector3 pointWorld0 = cameraToWorld(pointCam0, focal0, params0.pose); | ^~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:376:7: error: reference to ‘Vector3’ is ambiguous 376 | Vector3 pointWorld2 = cameraToWorld(pointCam2, focal2, params2.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:376:16: error: expected primary-expression before ‘>’ token 376 | Vector3 pointWorld2 = cameraToWorld(pointCam2, focal2, params2.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:376:18: error: ‘pointWorld2’ was not declared in this scope 376 | Vector3 pointWorld2 = cameraToWorld(pointCam2, focal2, params2.pose); | ^~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:378:7: error: reference to ‘Vector3’ is ambiguous 378 | Vector3 pointCam0To1 = worldToCamera(pointWorld0, focal1, params1.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:378:16: error: expected primary-expression before ‘>’ token 378 | Vector3 pointCam0To1 = worldToCamera(pointWorld0, focal1, params1.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:378:18: error: ‘pointCam0To1’ was not declared in this scope 378 | Vector3 pointCam0To1 = worldToCamera(pointWorld0, focal1, params1.pose); | ^~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:379:7: error: reference to ‘Vector3’ is ambiguous 379 | Vector3 pointCam2To1 = worldToCamera(pointWorld2, focal1, params1.pose); | ^~~~~~~ In file included from /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/Core:295, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/core/ParamsBase.h:8, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.h:7, from /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:3: /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:524:7: note: candidates are: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 524 | using Vector##SizeSuffix = Matrix<Type, Size, 1>;
    | ^~~~~~ /home/nadeem/anaconda3/envs/depth/include/eigen3/Eigen/src/Core/Matrix.h:541:1: note: in expansion of macro ‘EIGEN_MAKE_TYPEDEFS’ 541 | EIGEN_MAKE_TYPEDEFS(3, 3) | ^~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:34:7: note: ‘template using Vector3 = Eigen::Matrix<Type, 3, 1>’ 34 | using Vector3 = Matrix<T, 3, 1>; | ^~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:379:16: error: expected primary-expression before ‘>’ token 379 | Vector3 pointCam2To1 = worldToCamera(pointWorld2, focal1, params1.pose); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:379:18: error: ‘pointCam2To1’ was not declared in this scope 379 | Vector3 pointCam2To1 = worldToCamera(pointWorld2, focal1, params1.pose); | ^~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:381:18: error: request for member ‘x’ in ‘smoothLoss’, which is of non-class type ‘int’ 381 | smoothLoss.x() = | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:384:18: error: request for member ‘y’ in ‘smoothLoss’, which is of non-class type ‘int’ 384 | smoothLoss.y() = | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:394:20: error: request for member ‘z’ in ‘smoothLoss’, which is of non-class type ‘int’ 394 | smoothLoss.z() = repro0To1Disp + repro2To1Disp - disp1 * 2.0; | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:403:22: error: request for member ‘z’ in ‘smoothLoss’, which is of non-class type ‘int’ 403 | smoothLoss.z() = (maxDepth / minDepth - 1.0); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:405:22: error: request for member ‘z’ in ‘smoothLoss’, which is of non-class type ‘int’ 405 | smoothLoss.z() = log(minDepth / maxDepth); | ^ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp: In instantiation of ‘bool facebook::cp::{anonymous}::SceneFlowSmoothnessLoss::operator()(const T     const    , T    ) const [with T = double]’: /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:93:25: required from ‘bool ceres::DynamicAutoDiffCostFunction<CostFunctor, Stride>::Evaluate(const double const, double, double) const [with CostFunctor = facebook::cp::{anonymous}::SceneFlowSmoothnessLoss; int Stride = 4]’ /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:85:16: required from here /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:362:32: error: invalid conversion from ‘double’ to ‘int’ []8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-fpermissive-fpermissive]8;;] 362 | Map<Vector3> smoothLoss(residuals); | ^~~~~~~~~ | | | double* /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp: In instantiation of ‘bool facebook::cp::{anonymous}::SceneFlowSmoothnessLoss::operator()(const T* const*, T*) const [with T = ceres::Jet<double, 4>]’: /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:203:23: required from ‘bool ceres::DynamicAutoDiffCostFunction<CostFunctor, Stride>::Evaluate(const double* const*, double*, double**) const [with CostFunctor = facebook::cp::{anonymous}::SceneFlowSmoothnessLoss; int Stride = 4]’ /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:85:16: required from here /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:362:32: error: invalid conversion from ‘ceres::Jet<double, 4>’ to ‘int’ []8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-fpermissive-fpermissive]8;;] 362 | Map<Vector3> smoothLoss(residuals); | ^~~~~~~~~ | | | ceres::Jet<double, 4> /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp: In instantiation of ‘bool facebook::cp::{anonymous}::StaticSceneCost::operator()(const T* const*, T*) const [with T = double]’: /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:93:25: required from ‘bool ceres::DynamicAutoDiffCostFunction<CostFunctor, Stride>::Evaluate(const double* const*, double*, double**) const [with CostFunctor = facebook::cp::{anonymous}::StaticSceneCost; int Stride = 4]’ /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:85:16: required from here /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:259:39: error: ‘obsToCamera’ was not declared in this scope 259 | Vector3 pointCam0 = obsToCamera(obs0, params0); | ~~~~~~~~~~~^~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:261:22: error: ‘cameraToWorld’ was not declared in this scope 261 | cameraToWorld(pointCam0, focal0, params0.pose); | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:263:39: error: ‘obsToCamera’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation []8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-fpermissive-fpermissive]8;;] 263 | Vector3 pointCam1 = obsToCamera(obs1, params1); | ~~~~~~~~~~~^~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:270:45: error: ‘cameraToWorld’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation []8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-fpermissive-fpermissive]8;;] 270 | Vector3 pointWorld1 = cameraToWorld(pointCam1, focal1, params1.pose); | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:281:24: error: ‘worldToCamera’ was not declared in this scope 281 | worldToCamera(pointWorld0, focal1, params1.pose); | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp: In instantiation of ‘bool facebook::cp::{anonymous}::StaticSceneCost::operator()(const T const, T*) const [with T = ceres::Jet<double, 4>]’: /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:203:23: required from ‘bool ceres::DynamicAutoDiffCostFunction<CostFunctor, Stride>::Evaluate(const double* const*, double*, double**) const [with CostFunctor = facebook::cp::{anonymous}::StaticSceneCost; int Stride = 4]’ /home/nadeem/anaconda3/envs/depth/include/ceres/dynamic_autodiff_cost_function.h:85:16: required from here /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:259:39: error: ‘obsToCamera’ was not declared in this scope 259 | Vector3 pointCam0 = obsToCamera(*obs0, params0); | ~~~~~~~~~~~^~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:261:22: error: ‘cameraToWorld’ was not declared in this scope 261 | cameraToWorld(pointCam0, focal0, params0.pose); | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:263:39: error: ‘obsToCamera’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation []8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-fpermissive-fpermissive]8;;] 263 | Vector3 pointCam1 = obsToCamera(*obs1, params1); | ~~~~~~~~~~~^~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:270:45: error: ‘cameraToWorld’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation []8;;https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-fpermissive-fpermissive]8;;] 270 | Vector3 pointWorld1 = cameraToWorld(pointCam1, focal1, params1.pose); | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/PoseOptimizer.cpp:281:24: error: ‘worldToCamera’ was not declared in this scope 281 | worldToCamera(pointWorld0, focal1, params1.pose); | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ make[2]: Leaving directory '/home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/build' make[1]: Leaving directory '/home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/build' make: Leaving directory '/home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/build' make[2]: *** [CMakeFiles/lib_cvd.dir/build.make:216: CMakeFiles/lib_cvd.dir/PoseOptimizer.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:85: CMakeFiles/lib_cvd.dir/all] Error 2 make: *** [/home/nadeem/Downloads/robust_cvd/lib/cvd2/lib/build/Makefile:91: all] Error 2

    Process finished with exit code 2

    opened by muhammadnadeem23064 6
  • AttributeError: module 'cv2.cv2' has no attribute 'xfeatures2d'

    AttributeError: module 'cv2.cv2' has no attribute 'xfeatures2d' 

    ## Flow ##
Sampled 522 frame pairs.
Resizing flow to (224, 384)
Traceback (most recent call last):
  File "main.py", line 20, in <module>
    dp.process()
  File "/home/ubuntu/Downloads/cvd2/process.py", line 234, in process
    return self.pipeline()
  File "/home/ubuntu/Downloads/cvd2/process.py", line 171, in pipeline
    self.compute_flow()
  File "/home/ubuntu/Downloads/cvd2/process.py", line 135, in compute_flow
    self.flow.compute_flow(frame_pairs, self.params.flow_model)
  File "/home/ubuntu/Downloads/cvd2/flow.py", line 124, in compute_flow
    optical_flow_homography.process(args)
  File "/home/ubuntu/Downloads/cvd2/optical_flow_homography.py", line 269, in process
    flow = infer(args, OFlowModel, DEVICE, im1, im2)
  File "/home/ubuntu/Downloads/cvd2/optical_flow_homography.py", line 177, in infer
    img1, img2, img2_reg, H_BA = getimage(img1_name, img2_name)
  File "/home/ubuntu/Downloads/cvd2/optical_flow_homography.py", line 149, in getimage
    (kpsA, featuresA) = detectAndDescribe(frame1)
  File "/home/ubuntu/Downloads/cvd2/optical_flow_homography.py", line 69, in detectAndDescribe
    descriptor = cv2.xfeatures2d.SURF_create()
 AttributeError: module 'cv2.cv2' has no attribute 'xfeatures2d'

    >>> import cv2
>>> print(cv2.__version__)
4.5.2
>>> cv2.xfeatures2d.SURF_create()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'cv2.cv2' has no attribute 'xfeatures2d'
    opened by karliell 3
  • Is it possible?

    Is it possible?

    Is it possible to pause the Colab script execution after depth_midas2/depth get generated, substitute folder content by depth I've generated using Midas 3. And then continue processing? Or just upload my depth_midas2 folder and skip midas2 generation step? Is it possible to obtain a metric depth map using this network. Thank you in advance!

    opened by semel1 3
  • Main program running time is too long...

    Main program running time is too long...

    I try to run your code on google colab, but the running time for main.py is very long(more than 2 hours even using demo video), the program gets stuck in this position every time. Is this normal? I am not colab pro. image

    opened by zhuhu00 2
  • About downloading models

    About downloading models

    wget https://www.dropbox.com/s/v1oc39d8k5fa1zx/models.zip?dl=1 -O models.zip

    I tried many methods many times, such as wget and browser, but all failed to download the model. Is there any other download link? For example, Baidu Netdisk?

    opened by Minisal 2
  • Unable to download pretrained models

    Unable to download pretrained models

    Hi!

    The dropbox download link gives a 404 and the alternative download link needs a WeChat account for downloading the models.

    I kindly ask you to please share the models in a more straightforward way.

    Thank you and best regards!

    opened by AlverGant 1
  • How to get the true depth value for every pixel?

    How to get the true depth value for every pixel?

    I don't quite understand why you use

    https://github.com/facebookresearch/robust_cvd/blob/main/depth_fine_tuning.py#L269

    inv_depth_batch = 1.0 / depth_batch

            for i in range(batch_size):
                if batch_size > 1:
                    inv_depth = inv_depth_batch[i, :, :].squeeze()
                else:
                    inv_depth = inv_depth_batch
                frame_id = metadata["frame_id"][i]

                if frames is None or frame_id in frames:
                    image_io.save_raw_float32_image(
                        depth_fmt.format(frame_id) + ".raw", inv_depth
                    )

    why does inverse necessary? How to get true depth value? Thanks a lot for your help!!!!!!!!

    opened by karliell 1
  • Meaning of parameter `--frame_range 0-10`

    Meaning of parameter `--frame_range 0-10`

    Thanks for the amazing project! I'm wondering the meaning of this parameter: --frame_range 0-10

    In params.py file it says:

            self.parser.add_argument(
            "--frame_range", default="",
            type=frame_range.parse_frame_range,
            help="Range of depth to fine-tune, e.g., 0,2-10,21-40."
        )

    I set it to 10 and in the end I got 10 raw file in

    ./R0-10_hierarchical2_midas2/StD100.0_StR1.0_SmD0_SmR0.0/depth_e0000/e0000_filtered/depth/

    Does this mean it only processes the first 10 frames in the video? Is there any easy way to make it process the whole video (all frames) automatically? Thanks a lot!

    opened by karliell 1
  • Why images get downscaled?

    Why images get downscaled?

    Why all processed streams down scaled? Is there is a way to process and obtain depth in the same resolution as input source (without upscale processed images back to the original resolution)?

    opened by semel1 1
  • Problems in your previous version

    Problems in your previous version

    Hey there! I run your previous version of code and got half of the results and met this problem:

    https://github.com/facebookresearch/consistent_depth/issues/4#issuecomment-680177616

    May I ask could you please help me with this?

    opened by karliell 1
  • Adding Code of Conduct file

    Adding Code of Conduct file

    This is pull request was created automatically because we noticed your project was missing a Code of Conduct file.

    Code of Conduct files facilitate respectful and constructive communities by establishing expected behaviors for project contributors.

    This PR was crafted with love by Facebook's Open Source Team.

    CLA Signed 
    opened by facebook-github-bot 0
  • RTX Compatibility / PyTorch 1.8.0?

    RTX Compatibility / PyTorch 1.8.0?

    I'm having trouble getting RCVD to work with a RTX 3080 Ti card. It needs the PyTorch 1.8.0 in order to work but then RCVD doesn't work. Is there any way around this? Is it possible to use another version of PyTorch that works with RCVD and RTX 30 cards?

    opened by toooft 0
  • contrast loss

    contrast loss

    Hello I have noticed that there is an additional loss used by default in your algorithm, yet it wasn't described in the paper. It is defined in loss/contrast_loss.py andit can be disabled by setting "--lambda_contrast_loss 0". If I disable this loss, the final result appears to be extremely blurry. Could someone explain where this loss comes from and what it does? Was it taken from some other project? Thank you

    opened by lkosh 0
  • No Module Named 'lib_python' Error In Colab.

    No Module Named 'lib_python' Error In Colab.

    I managed to download Detectron2 using other methods than the ones provided and switched the OpenCV version from 3.4.2.16 to 3.4.2.17 since you can no longer install 16. Now I get no errors while installing everything, except that when I go to run the Robust CVD code, it tells me that lib_python is not installed. Here is the code: /content/cvd2 ['/content/cvd2', '/env/python', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/content/detectron2_repo', '/usr/lib/python3/dist-packages', '/content/cvd2/main.py', 'lib/build'] Traceback (most recent call last): File "main.py", line 12, in <module> from params import Video3dParamsParser File "/content/cvd2/params.py", line 8, in <module> from depth_fine_tuning import DepthFineTuningParams File "/content/cvd2/depth_fine_tuning.py", line 30, in <module> from loaders.video_dataset import VideoDataset, VideoFrameDataset File "/content/cvd2/loaders/video_dataset.py", line 16, in <module> from lib_python import (DepthVideo, ValueXformType, DepthXformType, SpatialXformType) ModuleNotFoundError: No module named 'lib_python'

    opened by FoxLynx64 4
  • How to use .raw files?

    How to use .raw files?

    Such a fantastic library, thanks a lot for your work!

    I need help with two things:

    1. I'm struggling to figure out how to open the .raw files and would like some help - I'm sure it's easy but I can't load them into any image software I own. How do I convert them to .png for instance?

    2. In the colab, what script language is the Result Visualization part made in? I can't seem to make it work on Ubuntu 20.04 LTS but I'm sure it's easy as well.

    opened by toooft 3
  • is it possible to run on mobile devices?

    is it possible to run on mobile devices?

    Dear all: I was shocked by the consistent/robust video depth estimation series work! it's so promising. But i also realised that the efficiency may be a strong obstable to overcome. anyone tested on this work give some advice about its efficiency? is it possible to run on mobile devices? thank you very much.

    opened by dragen1860 0
Owner
Facebook Research
Facebook Research
GitHub https://robust-cvd.github.io/
The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.

Geometric Structure Preserving Warp for Natural Image Stitching This repository contains our dataset and C++ implementation of the CVPR 2022 paper, Ge

null 21 Dec 22, 2022
The official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Graph Optimizer This repo contains the official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averagin

Chenyu 109 Dec 23, 2022
Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and deploy without Python.

Python Inference Script(PyIS) Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and d

Microsoft 13 Nov 4, 2022
A pytorch implementation of instant-ngp, as described in Instant Neural Graphics Primitives with a Multiresolution Hash Encoding.

torch-ngp A pytorch implementation of instant-ngp, as described in Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. Note: This

hawkey 1k Jan 6, 2023
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV Workshop @ CVPR 2021.

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022
Mixed reality VR laser tag using Oculus Quest 2 and OAK-D depth cameras. First prize winner for North America region in OpenCV AI Competition 2021.

Mixed Reality Laser Tag Copyright 2021 Bart Trzynadlowski Overview This is the source code to my Mixed Reality Laser Tag project, which won first priz

null 34 Jun 3, 2022
ORB-SLAM3-Monodepth is an extended version of ORB-SLAM3 that utilizes a deep monocular depth estimation network

ORB_SLAM3_Monodepth Introduction This repository was forked from [ORB-SLAM3] (https://github.com/UZ-SLAMLab/ORB_SLAM3). ORB-SLAM3-Monodepth is an exte

null 26 Jan 3, 2023
Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, SRMD, RealSR, Anime4K, RIFE, CAIN, DAIN and ACNet.

Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, SRMD, RealSR, Anime4K, RIFE, CAIN, DAIN and ACNet.

Aaron Feng 8.7k Dec 31, 2022
Implementation for the "Surface Reconstruction from 3D Line Segments" paper.

Surface Reconstruction from 3D Line Segments Surface reconstruction from 3d line segments. [Paper] [Supplementary Material] Langlois, P. A., Boulch, A

null 84 Dec 31, 2022
The official implementation of the research paper "DAG Amendment for Inverse Control of Parametric Shapes"

DAG Amendment for Inverse Control of Parametric Shapes This repository is the official Blender implementation of the paper "DAG Amendment for Inverse

Elie Michel 157 Dec 26, 2022
Fast and robust template matching with majority neighbour similarity and annulus projection transformation

A-MNS_TemplateMatching This is the official code for the PatternRecognition2020 paper: Fast and robust template matching with majority neighbour simil

Layjuns 22 Dec 30, 2022
New ultra super robust and fast programming language, fully supportable by G++ and Clang

Cplusplusplus New ultra super robust and fast programming language, fully supportable by G++ and Clang How to use: Just write #include <C+++.h> in you

Vladimir Melnikov 1 Nov 29, 2021
Fast and robust certifiable relative pose estimation

Fast and Robust Relative Pose Estimation for Calibrated Cameras This repository contains the code for the relative pose estimation between two central

null 42 Dec 6, 2022
Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation"

SSM-VLN Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation". Environment Installation Download Room-to-Room

hanqing 35 Dec 3, 2022
Codes for the paper "A Correlation Attack on Full SNOW-V and SNOW-Vi"

Codes for the paper "A Correlation Attack on Full SNOW-V and SNOW-Vi" #Contents Automatic search of linear approximation trails of SNOW-V Evaluating t

null 2 Oct 19, 2022
R3live - A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package

R3LIVE A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package News [Dec 31, 2021] Release of cod

HKU-Mars-Lab 1.3k Jan 4, 2023
Fast and robust face tracking addon for openFrameworks based on YOLO5Face

ofxFaceTracker3 Working in progress Fast and robust face tracking addon for openFrameworks based on YOLO5Face and ONNX Runtime. Features Fast and robu

Yuya Hanai 13 Nov 6, 2022
ICRA 2021 - Robust Place Recognition using an Imaging Lidar

Robust Place Recognition using an Imaging Lidar A place recognition package using high-resolution imaging lidar. For best performance, a lidar equippe

Tixiao Shan 296 Jan 1, 2023
This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicity.

Fast Face Classification (F²C) This is the code of our paper An Efficient Training Approach for Very Large Scale Face Recognition or F²C for simplicit

null 33 Jun 27, 2021