Tandem - [CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo

Related tags

Artificial Intelligence slam 3d-reconstruction visual-odometry multiview-stereo
Overview

TANDEM: Tracking and Dense Mapping
in Real-time using Deep Multi-view Stereo

Lukas Koestler1*    Nan Yang1,2*,†    Niclas Zeller2,3    Daniel Cremers1,2

*equal contribution    †corresponding author

1Technical University of Munich    2Artisense
3Karlsruhe University of Applied Sciences

Conference on Robot Learning (CoRL) 2021, London, UK

3DV 2021 Best Demo Award

arXiv | Video | OpenReview | Project Page

Code and Data

  • 📣 C++ code released before Christmas! Please check tandem/.
  • 📣 CVA-MVSNet released! Please check cva_mvsnet/.
  • 📣 Replica training data released! Please check replica/.
  • Minor improvements throughout January. Contributions are highly welcomed!
  • Docker image for TANDEM. Contributions are highly welcomed!

Abstract

In this paper, we present TANDEM a real-time monocular tracking and dense mapping framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of keyframes. To increase the robustness, we propose a novel tracking front-end that performs dense direct image alignment using depth maps rendered from a global model that is built incrementally from dense depth predictions. To predict the dense depth maps, we propose Cascade View-Aggregation MVSNet (CVA-MVSNet) that utilizes the entire active keyframe window by hierarchically constructing 3D cost volumes with adaptive view aggregation to balance the different stereo baselines between the keyframes. Finally, the predicted depth maps are fused into a consistent global map represented as a truncated signed distance function (TSDF) voxel grid. Our experimental results show that TANDEM outperforms other state-of-the-art traditional and learning-based monocular visual odometry (VO) methods in terms of camera tracking. Moreover, TANDEM shows state-of-the-art real-time 3D reconstruction performance.

Poster

Comments
  • No such file or directory

    No such file or directory

    When I make the project, I get

    fatal error: cub/block/block_reduce.cuh: No such file or directory
 #include <cub/block/block_reduce.cuh>

    but I find this .cuh file in my computer. I don't know how to do next, does anyone have the same question? My environment: Ubuntu18.04, CUDA10.2

    opened by Xie-HIT 6
  • Use preset=gui didn't get mesh

    Use preset=gui didn't get mesh

    Hi, I can successfully generate the mesh obj on the Euroc dataset. My problem is when I set preset=gui I can get some frames showing on a window but didn't see the mesh in the same window. Could you let me know the culprit? Thanks. image

    opened by mathmax12 6
  • Cudatoolkit couldn't find

    Cudatoolkit couldn't find

    Hi,

    I installed CUDAToolkit 11.1.0 as per the nvidia official website. When I ran nvcc -V, the terminal displayed the text in the first

    image. I believe It means the installation is complete. However, when I ran 'cmake ..', an error happened(the second image).

    Can anyone help me solve this problem?

    Thank you! image

    image

    opened by Bobkk-k 5
  • make error related to Sophus

    make error related to Sophus

    Hi, I meet some problem related to sophus while building the project, the log is : [ 81%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/dense_tracking_config.cpp.o [ 83%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/util/histogram.cpp.o [ 84%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/dense_tracking_impl.cpp.o [ 86%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/OpenCV/ImageDisplay_OpenCV.cpp.o [ 88%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/KeyFrameDisplay.cpp.o [ 89%] Building CXX object CMakeFiles/tandem.dir/src/dvo-core/src/util/id_generator.cpp.o [ 91%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/DrFrameDisplay.cpp.o [ 93%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/OpenCV/ImageRW_OpenCV.cpp.o [ 94%] Building CXX object CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp.o In file included from /home/hutiancheng/project/tandem/tandem/src/util/settings.cpp:27:0: /usr/local/include/boost/bind.hpp:41:1: note: #pragma message: The practice of declaring the Bind placeholders (1, 2, ...) in the global namespace is deprecated. Please use <boost/bind/bind.hpp> + using namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to retain the current behavior. ) ^ [ 94%] Built target dr_mvsnet_test /home/hutiancheng/project/tandem/tandem/src/FullSystem/HessianBlocks.cpp: In member function ‘void dso::FrameHessian::setStateZero(const Vec10&)’: /home/hutiancheng/project/tandem/tandem/src/FullSystem/HessianBlocks.cpp:84:23: error: ‘template<class Scalar, int Options> class Sophus::SE3’ used without template parameters SE3 EepsP = Sophus::SE3::exp(eps); ^~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/HessianBlocks.cpp:85:23: error: ‘template<class Scalar, int Options> class Sophus::SE3’ used without template parameters SE3 EepsM = Sophus::SE3::exp(-eps); ^~~ /home/hutiancheng/project/tandem/tandem/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp: In member function ‘void dso::IOWrap::PangolinDSOViewer::run()’: /home/hutiancheng/project/tandem/tandem/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp:280:27: error: ‘glDrawColoredVertices’ is not a member of ‘pangolin’ pangolin::glDrawColoredVertices( ^~~~~~~~~~~~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp:280:27: note: suggested alternative: ‘glDrawColouredCube’ pangolin::glDrawColoredVertices( ^~~~~~~~~~~~~~~~~~~~~ glDrawColouredCube /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp: In member function ‘dso::Vec4 dso::FullSystem::trackNewCoarse(dso::FrameHessian*)’: /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:452:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, 0, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:452:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:455:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:455:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:458:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, 0, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:458:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:461:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, 0, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:461:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:464:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, -rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:464:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:467:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, 0, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:467:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:470:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:470:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:473:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:473:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:476:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, 0, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:476:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:479:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:479:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:482:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, -rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:482:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:485:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, 0, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:485:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:488:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, -rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:488:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:491:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:491:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:494:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, 0, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:494:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:497:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, -rotDelta, 0), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:497:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:500:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, 0, -rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:500:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:503:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, 0, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:503:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:506:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, -rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:506:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:509:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, -rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:509:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:512:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:512:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:515:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, -rotDelta, rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:515:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:518:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, -rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:518:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:521:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, -rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:521:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:524:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, rotDelta, -rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:524:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:527:46: error: ‘Quaterniond’ is not a member of ‘Sophus’ SE3(Sophus::Quaterniond(1, rotDelta, rotDelta, rotDelta), ^~~~~~~~~~~ /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:527:46: note: suggested alternative: In file included from /usr/local/include/eigen3/Eigen/Geometry:42:0, from /usr/local/include/eigen3/Eigen/Dense:6, from /usr/local/include/sophus/rotation_matrix.hpp:7, from /usr/local/include/sophus/so3.hpp:7, from /usr/local/include/sophus/rxso3.hpp:7, from /usr/local/include/sophus/sim3.hpp:7, from /home/hutiancheng/project/tandem/tandem/src/util/NumType.h:28, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.h:29, from /home/hutiancheng/project/tandem/tandem/src/FullSystem/FullSystem.cpp:32: /usr/local/include/eigen3/Eigen/src/Geometry/Quaternion.h:310:28: note: ‘Eigen::Quaterniond’ typedef Quaternion Quaterniond; ^~~~~~~~~~~ CMakeFiles/tandem.dir/build.make:209: recipe for target 'CMakeFiles/tandem.dir/src/FullSystem/HessianBlocks.cpp.o' failed make[2]: *** [CMakeFiles/tandem.dir/src/FullSystem/HessianBlocks.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... CMakeFiles/tandem.dir/build.make:573: recipe for target 'CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp.o' failed make[2]: *** [CMakeFiles/tandem.dir/src/IOWrapper/Pangolin/PangolinDSOViewer.cpp.o] Error 1 CMakeFiles/tandem.dir/build.make:79: recipe for target 'CMakeFiles/tandem.dir/src/FullSystem/FullSystem.cpp.o' failed make[2]: *** [CMakeFiles/tandem.dir/src/FullSystem/FullSystem.cpp.o] Error 1 CMakeFiles/Makefile2:190: recipe for target 'CMakeFiles/tandem.dir/all' failed make[1]: *** [CMakeFiles/tandem.dir/all] Error 2 Makefile:100: recipe for target 'all' failed make: *** [all] Error 2

    could you give me some device? Thanks a lot!

    opened by autohtc92 4
  • cmake error

    cmake error

    -- The C compiler identification is GNU 8.4.0 -- The CXX compiler identification is GNU 8.4.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "2.91.0") -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
    -- Found Boost: /usr/include (found version "1.67.0") found components: system thread chrono date_time atomic -- Found CUDA: /usr/local/cuda-11.1 (found version "11.1") -- Caffe2: CUDA detected: 11.1 -- Caffe2: CUDA nvcc is: /usr/local/cuda-11.1/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.1 -- Caffe2: Header version is: 11.1 -- Found CUDNN: /usr/local/cuda-11.1/lib64/libcudnn.so
    -- Found cuDNN: v7.6.5 (include: /usr/local/cuda-11.1/include, library: /usr/local/cuda-11.1/lib64/libcudnn.so) -- /usr/local/cuda-11.1/lib64/libnvrtc.so shorthash is 3a20f2b6 -- Autodetected CUDA architecture(s): 7.5 -- Added CUDA NVCC flags for: -gencode;arch=compute_75,code=sm_75 -- Found Torch: /home/haimou/文档/libtorch-cxx11-abi-shared-with-deps-1.9.0+cu111/libtorch/lib/libtorch.so
    -- Found OpenCV: /usr/local (found version "4.2.0") -- The CUDA compiler identification is NVIDIA 11.1.105 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/local/cuda-11.1/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Found CUDAToolkit: /usr/local/cuda-11.1/include (found version "11.1.105") -- Found Eigen headers in: /usr/local/include/eigen3 -- Release mode (!= not Debug). Adding -O3;-lineinfo to NVCC FLAGS --- found PANGOLIN, compiling tandem_pangolin library. --- found OpenCV (/usr/local/include/opencv4), compiling tandem_opencv library. --- compiling tandem_dataset. --- not building tandem_demo, since either don't have openCV, Pangolin, or Realsense. --- not building realsense_calib_recorder, since either don't have openCV, Pangolin, or Realsense. -- Configuring done CMake Error at CMakeLists.txt:131 (add_executable): Target "tandem_dataset" links to target "Eigen3::Eigen" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at CMakeLists.txt:131 (add_executable): Target "tandem_dataset" links to target "Eigen3::Eigen" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    -- Generating done CMake Generate step failed. Build files cannot be regenerated correctly.

    opened by Ruinmou 4
  • Error with c++11 compiler

    Error with c++11 compiler

    Hello!

    I am trying to run the code, while building tandem I am getting 2 major errors from cuda_coarse_tracker side. I have cuda 11.1, and same cudNN and libtorch version as mentioned in readme.

    (tandem-exp) [email protected]:~/tandem/tandem/build$ make -jConsolidate compiler generated dependencies of target dr-mvsnet
[  1%] Building CXX object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o
[  3%] Building CUDA object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o
Consolidate compiler generated dependencies of target dr-fusion
[  6%] Built target dr-mvsnet
[ 22%] Built target dr-fusion
Consolidate compiler generated dependencies of target dr_mvsnet_test
[ 25%] Built target dr_mvsnet_test
/home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9:18: fatal error: cnpy.h: No such file or directory
compilation terminated.
libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:75: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o' failed
make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /usr/include/c++/5/type_traits:35:0,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/utils.h:11,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/numeric_cuda.h:16,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker_private.cu:4:
/usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support \
  ^
libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:89: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o' failed
make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o] Error 1
CMakeFiles/Makefile2:274: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all' failed
make[1]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
(tandem-exp) [email protected]:~/tandem/tandem/build$

    Could you suggest anything to solve these?

    I am using SET(CMAKE_CXX_FLAGS "-std=c++0x") in CMakeLists.txt

    Thanks,

    Update1: I fixed the cnpy error by cloning the cnpy repo, cmake, make and make install for future reference

    C++11 Compiler error still remains

    (tandem-exp) [email protected]:~/tandem/tandem/build$ cmake ..     -DCMAKE_BUILD_TYPE=Release     -DCMAKE_PREFIX_PATH=$TANDEM_LIBTORCH_DIR     -DCUDNN_LIBRARY=$TANDEM_CUDNN_LIBRARY     -DCUDNN_INCLUDE_PATH=$TANDEM_CUDNN_INCLUDE_PATH
-- Caffe2: CUDA detected: 11.1
-- Caffe2: CUDA nvcc is: /usr/local/cuda-11.1/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.1
-- Caffe2: Header version is: 11.1
-- Found cuDNN: v8.0.5  (include: /usr/local/cuda/include, library: /usr/local/cuda-11.1/lib64/libcudnn.so)
-- /usr/local/cuda-11.1/lib64/libnvrtc.so shorthash is 3a20f2b6
-- Autodetected CUDA architecture(s):  6.1
-- Added CUDA NVCC flags for: -gencode;arch=compute_61,code=sm_61
-- Found Eigen headers in: /usr/local/include/eigen3
-- Release mode (!= not Debug). Adding -O3;-lineinfo to NVCC FLAGS
--- found PANGOLIN, compiling tandem_pangolin library.
--- found OpenCV (/usr/local/include;/usr/local/include/opencv), compiling tandem_opencv library.
--- compiling tandem_dataset.
--- not building tandem_demo, since either don't have openCV, Pangolin, or Realsense.
--- not building realsense_calib_recorder, since either don't have openCV, Pangolin, or Realsense.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vanjani/tandem/tandem/build
(tandem-exp) [email protected]:~/tandem/tandem/build$ make -jConsolidate compiler generated dependencies of target dr-mvsnet
Consolidate compiler generated dependencies of target dr-fusion
[  1%] Building CXX object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o
[  3%] Building CUDA object libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o
[ 18%] Built target dr-fusion
[ 22%] Built target dr-mvsnet
Consolidate compiler generated dependencies of target dr_mvsnet_test
[ 25%] Built target dr_mvsnet_test
In file included from /usr/include/c++/5/type_traits:35:0,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/utils.h:11,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/include/private/numeric_cuda.h:16,
                 from /home/vanjani/tandem/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker_private.cu:4:
/usr/include/c++/5/bits/c++0x_warning.h:32:2: error: #error This file requires compiler and library support for the ISO C++ 2011 standard. This support must be enabled with the -std=c++11 or -std=gnu++11 compiler options.
 #error This file requires compiler and library support \
  ^
libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:89: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o' failed
make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker_private.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:274: recipe for target 'libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all' failed
make[1]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2
(tandem-exp) [email protected]:~/tandem/tandem/build$

    This error was coming in dr_mvsnet too but using SET(CMAKE_CXX_FLAGS "-std=c++0x") in Cmakelists.txt solved the issue but somehow it is not helping for cuda_coarse_tracler

    opened by pankhurivanjani 3
  • links to target

    links to target "CUDA::cublas" but the target was not found.

    Hi， when I run

    cmake .. \
   -DCMAKE_BUILD_TYPE=Release \
   -DCMAKE_PREFIX_PATH=$TANDEM_LIBTORCH_DIR \
   -DCUDNN_LIBRARY=$TANDEM_CUDNN_LIBRARY \
   -DCUDNN_INCLUDE_PATH=$TANDEM_CUDNN_INCLUDE_PATH

    there are some errors: -- The C compiler identification is GNU 7.5.0 -- The CXX compiler identification is GNU 7.5.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE
    -- Boost version: 1.65.1 -- Found the following Boost libraries: -- system -- thread -- chrono -- date_time -- atomic -- Found CUDA: /usr/local/cuda (found version "11.1") -- Caffe2: CUDA detected: 11.1 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 11.1 -- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so
    -- Found cuDNN: v8.2.1 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so) -- /usr/local/cuda/lib64/libnvrtc.so shorthash is 3a20f2b6 -- Autodetected CUDA architecture(s): 8.6 -- Added CUDA NVCC flags for: -gencode;arch=compute_86,code=sm_86 -- Found Torch: /home/libtorch-cxx11-abi-shared-with-deps-1.9.0+cu111/libtorch/lib/libtorch.so
    -- Found OpenCV: /usr/local (found version "4.5.4") -- Boost version: 1.65.1 -- Found the following Boost libraries: -- system -- thread -- chrono -- date_time -- atomic -- The CUDA compiler identification is NVIDIA 11.1.105 -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc -- works -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Found Eigen headers in: /usr/include/eigen3 -- Release mode (!= not Debug). Adding -O3;-lineinfo to NVCC FLAGS --- found PANGOLIN, compiling tandem_pangolin library. --- found OpenCV (/usr/local/include/opencv4), compiling tandem_opencv library. --- compiling tandem_dataset. -- TANDEM DEMO will be compiled -- Realsense Calib Recorded Will be compiled -- Configuring done CMake Error at CMakeLists.txt:115 (add_library): Target "tandem" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at CMakeLists.txt:142 (add_executable): Target "tandem_demo" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at CMakeLists.txt:131 (add_executable): Target "tandem_dataset" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error at libdr/cuda_coarse_tracker/CMakeLists.txt:27 (add_library): Target "cuda_coarse_tracker" links to target "CUDA::cublas" but the target was not found. Perhaps a find_package() call is missing for an IMPORTED target, or an ALIAS target is missing?

    CMake Error: Error required internal CMake variable not set, cmake may not be built correctly. Missing variable is: CMAKE_CUDA_DEVICE_LINK_EXECUTABLE CMake Error: Error required internal CMake variable not set, cmake may not be built correctly. Missing variable is: CMAKE_CUDA_DEVICE_LINK_EXECUTABLE -- Generating done

    how to fix it ? Thanks!

    opened by DormouseLi 3
  • Segmentation fault when testing tandem on a custom synthetic data

    Segmentation fault when testing tandem on a custom synthetic data

    Hello,

    I am trying to test Tandem on a custom dataset.

    When it loads the model the program crashes and I get a segmentation fault like this

    (tandem) [email protected]:~/tandem/tandem$ ./build/bin/tandem_dataset preset=gui result_folder=/home/pankhuri/tandem-dataset/results files=/home/pankhuri/tandem-dataset/images calib=/home/pankhuri/tandem-dataset/camera.txt mvsnet_folder=/home/pankhuri/tandem/tandem/exported/tandem mode=1
loading data from /home/pankhuri/tandem-dataset/images!
loading calibration from /home/pankhuri/tandem-dataset/camera.txt!
Loading MVSNet from /home/pankhuri/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
	Setting 'gui':
	- no real-time enforcing
	- 2000 active points
	- 5-7 active frames
	- 1-6 LM iteration each KF
	- TSDF fusion: yes
	- dense tracking on cpu (step=1)
	- Pangolin
	  - Fullscreen: 0
	  - Mesh: 1
	  - Smaller Images: 1

Reading Calibration from file /home/pankhuri/tandem-dataset/camera.txt ... found!
Input resolution: 640 480
In: 300.000000 300.000000 480.000000 270.000000 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -1.6159 - 0.5352; y: -0.9090 - 0.7036!
iteration 00001: range: x: -1.6078 - 0.5325; y: -0.9090 - 0.7036!
iteration 00002: range: x: -1.5998 - 0.5299; y: -0.9090 - 0.7036!
iteration 00003: range: x: -1.5998 - 0.5299; y: -0.9045 - 0.7000!
iteration 00004: range: x: -1.5998 - 0.5299; y: -0.8999 - 0.6965!
iteration 00005: range: x: -1.5998 - 0.5299; y: -0.8999 - 0.6965!

Rectified Kamera Matrix:
300.051       0 480.015
      0 300.035 270.011
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
got 0 images and 0 timestamps and 0 exposures.!
ImageFolderReader: got 0 files in /home/pankhuri/tandem-dataset/images!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!
START PANGOLIN!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
	Depth correct     : 0, error: 0.0106286
	Confidence correct: 1, error: 0.00296127
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.0029548
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Performance:
	CallAsync     : 6.832 ms
	Ready         : 0 ms
	GetResult     : 116.539 ms
There has been an error. Do not use the model.
Segmentation fault (core dumped)
(tandem) [email protected]:~/tandem/tandem$

    What could be the reason behind this segmentation fault? Is it dependent on geometric calibration?

    Note:

    It works fine with other dataset like tum rgbd freiburg sequence, although there comes a warning of not loading MVSNet successfully

    (tandem) [email protected]:~/tandem/tandem$ ./build/bin/tandem_dataset preset=gui result_folder=/home/pankhuri/tandem-dataset/results files=/home/pankhuri/rgbd_dataset_freiburg2_xyz/rgb calib=/home/pankhuri/rgbd_dataset_freiburg2_xyz/camera.txt mvsnet_folder=/home/pankhuri/tandem/tandem/exported/tandem mode=1
loading data from /home/pankhuri/rgbd_dataset_freiburg2_xyz/rgb!
loading calibration from /home/pankhuri/rgbd_dataset_freiburg2_xyz/camera.txt!
Loading MVSNet from /home/pankhuri/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
	Setting 'gui':
	- no real-time enforcing
	- 2000 active points
	- 5-7 active frames
	- 1-6 LM iteration each KF
	- TSDF fusion: yes
	- dense tracking on cpu (step=1)
	- Pangolin
	  - Fullscreen: 0
	  - Mesh: 1
	  - Smaller Images: 1

Reading Calibration from file /home/pankhuri/rgbd_dataset_freiburg2_xyz/camera.txt ... found!
Input resolution: 640 480
In: 517.300000 516.500000 318.600000 255.300000 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -0.6220 - 0.6255; y: -0.4991 - 0.4374!
iteration 00001: range: x: -0.6188 - 0.6224; y: -0.4991 - 0.4374!
iteration 00002: range: x: -0.6158 - 0.6193; y: -0.4991 - 0.4374!
iteration 00003: range: x: -0.6158 - 0.6193; y: -0.4966 - 0.4352!
iteration 00004: range: x: -0.6158 - 0.6193; y: -0.4942 - 0.4331!
iteration 00005: range: x: -0.6158 - 0.6193; y: -0.4942 - 0.4331!

Rectified Kamera Matrix:
517.406       0 318.595
      0 516.592 255.281
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
set EXPOSURES to zero!
got 3669 images and 3669 timestamps and 0 exposures.!
ImageFolderReader: got 3669 files in /home/pankhuri/rgbd_dataset_freiburg2_xyz/rgb!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!
START PANGOLIN!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
	Depth correct     : 0, error: 0.0106286
	Confidence correct: 1, error: 0.00296125
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295481
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.0029548
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Correctness:
	Depth correct     : 0, error: 0.0106182
	Confidence correct: 1, error: 0.00295482
Performance:
	CallAsync     : 6.80075 ms
	Ready         : 0 ms
	GetResult     : 116.749 ms
There has been an error. Do not use the model.
Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER (2048 pts)!

TANDEM TIMING: ==================
3668 Frames (32.8 fps)
30.48ms per frame; 
111.79s total time; 
======================

DrFusion::SaveMeshToFile volume_->ExtractMesh (485 ms)
DrFusion::SaveMeshToFile mesh.SaveToFile (3587 ms)
Mesh Saving done!
^CCaught signal 2
    opened by pankhurivanjani 2
  • make error about Sophus

    make error about Sophus

    make error about Sophus

    image

    When I fellow the readme in the Sophus folder to make Sophus, something that looks likes make fail appeared.

    image

    Does anyone know how to solve it ? Thanks!

    opened by Bobkk-k 2
  • not able to open

    not able to open "tandem/cva_mvsnet/model.pt"

    Thanks for the opensourcing code. It seems the model.pt is not provided for running your system on euroc dataset.

    After running the command for the given dataset euroc, the system seems to initialize successfully but is not able to open the "model.pt" using cva_mvsnet. The flag "mvsnet_folder=Folder that contains the exported MVSNet." is not very clearly explained. Following is my log from the terminal:

    ~/tandem/tandem$ ./build/bin/tandem_dataset preset=dataset result_folder=./results files=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images calib=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/shu/Examples/tandem/cva_mvsnet mode=1

    loading data from /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images! loading calibration from /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt! Loading MVSNet from /home/shu/Examples/tandem/cva_mvsnet/! PHOTOMETRIC MODE WITHOUT CALIBRATION!

    =============== TANDEM Settings: =============== Setting 'dataset': - no real-time enforcing - 2000 active points - 5-7 active frames - 1-6 LM iteration each KF - TSDF fusion: yes - dense tracking on cpu (step=1) - Pangolin - Fullscreen: 0 - Mesh: 1 - Smaller Images: 1

    Reading Calibration from file /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt ... found! Input resolution: 640 480 In: 436.225021 436.225021 310.162763 256.946041 0.000000 Out: Rectify Crop Output resolution: 640 480 finding CROP optimal new model! initial range: x: -0.7181 - 0.7613; y: -0.5949 - 0.5141! iteration 00001: range: x: -0.7145 - 0.7575; y: -0.5949 - 0.5141! iteration 00002: range: x: -0.7109 - 0.7537; y: -0.5949 - 0.5141! iteration 00003: range: x: -0.7109 - 0.7537; y: -0.5919 - 0.5115! iteration 00004: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090! iteration 00005: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!

    Rectified Kamera Matrix: 436.27 0 310.165 0 436.28 256.95 0 0 1

    NO PHOTOMETRIC Calibration! Reading Photometric Calibration from file PhotometricUndistorter: Could not open file! set timestamps and exposures to zero! set EXPOSURES to zero! got 2778 images and 0 timestamps and 0 exposures.! ImageFolderReader: got 2778 files in /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images! using pyramid levels 0 to 3. coarsest resolution: 80 x 60!

    ----DRMVSNET Initalizing fusion----

    ----DRMVSNET Initalizing fusion done---- DrMvsnet torch::cuda::is_vailable == true --> seems good terminate called after throwing an instance of 'c10::Error' what(): open file failed, file path: /home/shu/Examples/tandem/cva_mvsnet/model.pt Exception raised from FileAdapter at ../caffe2/serialize/file_adapter.cc:11 (most recent call first): frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x69 (0x7f8ea507c1d9 in /home/shu/Examples/libtorch/lib/libc10.so) frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0xd2 (0x7f8ea5078812 in /home/shu/Examples/libtorch/lib/libc10.so) frame #2: caffe2::serialize::FileAdapter::FileAdapter(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x1ff (0x7f8ea7ddbc0f in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #3: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >&) + 0x40 (0x7f8ea92e4b50 in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #4: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device) + 0x6f (0x7f8ea92e4cdf in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #5: + 0x16b827 (0x55a76f9e3827 in ./build/bin/tandem_dataset) frame #6: + 0x163482 (0x55a76f9db482 in ./build/bin/tandem_dataset) frame #7: + 0x3a6bc (0x55a76f8b26bc in ./build/bin/tandem_dataset) frame #8: + 0x2980d (0x55a76f8a180d in ./build/bin/tandem_dataset) frame #9: + 0xbd6df (0x7f8e526046df in /usr/lib/x86_64-linux-gnu/libstdc++.so.6) frame #10: + 0x76db (0x7f8ea55006db in /lib/x86_64-linux-gnu/libpthread.so.0) frame #11: clone + 0x3f (0x7f8e51cc171f in /lib/x86_64-linux-gnu/libc.so.6)

    Aborted (core dumped)

    opened by PeterFWS 2
  • RuntimeError: CUDA out of memory

    RuntimeError: CUDA out of memory

    I will like to start by saying thank you for the amazing project and paper! After I cloned the repository I followed the instructions to set up the environment as described in the read me file. Finally, I downloaded the replica dataset and started the training, but before it even starts it exists with the CUDA out of memory error message. I'm using NVIDIA GeForce RTX 2080 Ti. I will put the full error message below. Any help would be appreciated . Thank you !!

    
GPU available: True, used: True
INFO:lightning:GPU available: True, used: True
No environment variable for node rank defined. Set as 0.
WARNING:lightning:No environment variable for node rank defined. Set as 0.
CUDA_VISIBLE_DEVICES: [0]
INFO:lightning:CUDA_VISIBLE_DEVICES: [0]
Epoch 1:   0%|                                                                                                                                                                     | 0/2714 [00:00<?, ?it/s]Traceback (most recent call last):
  File "train.py", line 128, in <module>
    main(*parse_args(parser))
  File "train.py", line 124, in main
    trainer.fit(model)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 859, in fit
    self.single_gpu_train(model)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/distrib_parts.py", line 503, in single_gpu_train
    self.run_pretrain_routine(model)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1015, in run_pretrain_routine
    self.train()
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 347, in train
    self.run_training_epoch()
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 419, in run_training_epoch
    _outputs = self.run_training_batch(batch, batch_idx)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 597, in run_training_batch
    loss, batch_output = optimizer_closure()
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 561, in optimizer_closure
    output_dict = self.training_forward(split_batch, batch_idx, opt_idx, self.hiddens)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py", line 727, in training_forward
    output = self.model.training_step(*args)
  File "/home/tmc/tandem/cva_mvsnet/models/tandem.py", line 101, in training_step
    outputs = self(batch)
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/tmc/tandem/cva_mvsnet/models/tandem.py", line 43, in forward
    depth_max=batch['depth_max'],
  File "/home/tmc/miniconda3/envs/tandem/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/tmc/tandem/cva_mvsnet/models/cva_mvsnet.py", line 162, in forward
    volume_gate=self.volume_gates[stage] if self.view_aggregation else None,
  File "/home/tmc/tandem/cva_mvsnet/models/module.py", line 1085, in depth_prediction
    half_pixel_centers=half_pixel_centers
  File "/home/tmc/tandem/cva_mvsnet/models/module.py", line 891, in homo_warping
    warped_src_fea[torch.isnan(warped_src_fea)] = 0
RuntimeError: CUDA out of memory. Tried to allocate 2.34 GiB (GPU 0; 10.76 GiB total capacity; 7.43 GiB already allocated; 740.06 MiB free; 8.94 GiB reserved in total by PyTorch)
Epoch 1:   0%|          | 0/2714 [00:04<?, ?it/s]
    opened by yohannes-taye 2
  • Saving sparse reconstruction

    Saving sparse reconstruction

    Hi. I wanted to ask how we can save the Sparse reconstruction instead of normal one. I nthe GUI we have option show sparse but I am not sure how to save that sparse map.

    opened by pytholic 0
  • Segmentation fault

    Segmentation fault

    Hi and thank you for the repo. I keep getting the following error on 80% of the videos. I have tried different devices like gopro, iphone, drone footage. I also performed checkerboard calibration.

    There has been an error. Do not use the model.
Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER (1961 pts)!
I THINK INITIALIZATINO FAILED! Resetting.
RESETTING!
destroyed ThreadReduce
INITIALIZE FROM INITIALIZER (1961 pts)!
Segmentation fault (core dumped)
    opened by pytholic 0
  • Docker and bash support

    Docker and bash support

    Hi. I tested the dockerfile provided by rancheng in an earlier pull request. It works. However, there were some shortcomings.

    I have created a detailed repository with updated instructions, docker support including the docker image and added bash script for faster map generation. It can be found here.

    I am creating this pull request so that you can check it and include it in the main repository,

    Cheers!

    opened by pytholic 0
  • cnpy error when i install

    cnpy error when i install

    when i run make -j i got :

    what should i do to fix it?

    /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp: In member function 'Vec6 CudaCoarseTracker::calcRes(const Eigen::Matrix&, float, const Vector2d&, float)': /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:218:64: error: no matching function for call to 'npy_save(const char [25], float [6], )' 218 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:218:64: note: candidate expects 6 arguments, 3 provided 218 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:221:87: error: no matching function for call to 'npy_save(const char [48], float [6], )' 221 | cnpy::npy_save("cct_data/setting_huberTH_maxEnergy_cutoffTH.npy", nfxfyaffLLb, {3}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:221:87: note: candidate expects 6 arguments, 3 provided 221 | cnpy::npy_save("cct_data/setting_huberTH_maxEnergy_cutoffTH.npy", nfxfyaffLLb, {3}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:223:79: error: no matching function for call to 'npy_save(const char [30], float*&, )' 223 | cnpy::npy_save("cct_data/refToNew_Ki_host.npy", refToNew_Ki_host, {16 + 9}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:223:79: note: candidate expects 6 arguments, 3 provided 223 | cnpy::npy_save("cct_data/refToNew_Ki_host.npy", refToNew_Ki_host, {16 + 9}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:227:65: error: no matching function for call to 'npy_save(const char [18], float*&, )' 227 | cnpy::npy_save("cct_data/pc_u.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:227:65: note: candidate expects 6 arguments, 3 provided 227 | cnpy::npy_save("cct_data/pc_u.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:230:65: error: no matching function for call to 'npy_save(const char [18], float*&, )' 230 | cnpy::npy_save("cct_data/pc_v.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:230:65: note: candidate expects 6 arguments, 3 provided 230 | cnpy::npy_save("cct_data/pc_v.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:233:70: error: no matching function for call to 'npy_save(const char [23], float*&, )' 233 | cnpy::npy_save("cct_data/pc_idepth.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:233:70: note: candidate expects 6 arguments, 3 provided 233 | cnpy::npy_save("cct_data/pc_idepth.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:236:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 236 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:236:69: note: candidate expects 6 arguments, 3 provided 236 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:241:74: error: no matching function for call to 'npy_save(const char [19], float*&, )' 241 | cnpy::npy_save("cct_data/dInew.npy", buf, {(unsigned long) 3 * w * h}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:241:74: note: candidate expects 6 arguments, 3 provided 241 | cnpy::npy_save("cct_data/dInew.npy", buf, {(unsigned long) 3 * w * h}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp: In member function 'void CudaCoarseTracker::calcG(Eigen::Matrix&, Eigen::Matrix&, float, const Vector2d&)': /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:290:64: error: no matching function for call to 'npy_save(const char [25], float [6], )' 290 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:290:64: note: candidate expects 6 arguments, 3 provided 290 | cnpy::npy_save("cct_data/nfxfyaffLLb.npy", nfxfyaffLLb, {6}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:294:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 294 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:294:69: note: candidate expects 6 arguments, 3 provided 294 | cnpy::npy_save("cct_data/pc_color.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:297:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 297 | cnpy::npy_save("cct_data/warped_u.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:297:69: note: candidate expects 6 arguments, 3 provided 297 | cnpy::npy_save("cct_data/warped_u.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:300:69: error: no matching function for call to 'npy_save(const char [22], float*&, )' 300 | cnpy::npy_save("cct_data/warped_v.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:300:69: note: candidate expects 6 arguments, 3 provided 300 | cnpy::npy_save("cct_data/warped_v.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:303:70: error: no matching function for call to 'npy_save(const char [23], float*&, )' 303 | cnpy::npy_save("cct_data/warped_dx.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:303:70: note: candidate expects 6 arguments, 3 provided 303 | cnpy::npy_save("cct_data/warped_dx.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:306:70: error: no matching function for call to 'npy_save(const char [23], float*&, )' 306 | cnpy::npy_save("cct_data/warped_dy.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:306:70: note: candidate expects 6 arguments, 3 provided 306 | cnpy::npy_save("cct_data/warped_dy.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:309:74: error: no matching function for call to 'npy_save(const char [27], float*&, )' 309 | cnpy::npy_save("cct_data/warped_idepth.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:309:74: note: candidate expects 6 arguments, 3 provided 309 | cnpy::npy_save("cct_data/warped_idepth.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:312:76: error: no matching function for call to 'npy_save(const char [29], float*&, )' 312 | cnpy::npy_save("cct_data/warped_residual.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:312:76: note: candidate expects 6 arguments, 3 provided 312 | cnpy::npy_save("cct_data/warped_residual.npy", buf, {(unsigned long) n}); | ^ /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:315:74: error: no matching function for call to 'npy_save(const char [27], float*&, )' 315 | cnpy::npy_save("cct_data/warped_weight.npy", buf, {(unsigned long) n}); | ^ In file included from /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:9: /usr/local/include/cnpy.h:64:31: note: candidate: 'template void cnpy::npy_save(std::string, const T*, const unsigned int*, unsigned int, std::string, std::string)' 64 | template void npy_save(std::string fname, const T* data, const unsigned int* shape, const unsigned int ndims, std::string mode = "w", std::string order = "C") { | ^~~~~~~~ /usr/local/include/cnpy.h:64:31: note: template argument deduction/substitution failed: /home/tandem/tandem-master/tandem/libdr/cuda_coarse_tracker/src/cuda_coarse_tracker.cpp:315:74: note: candidate expects 6 arguments, 3 provided 315 | cnpy::npy_save("cct_data/warped_weight.npy", buf, {(unsigned long) n}); | ^ make[2]: *** [libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/build.make:76: libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/src/cuda_coarse_tracker.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[1]: *** [CMakeFiles/Makefile2:275: libdr/cuda_coarse_tracker/CMakeFiles/cuda_coarse_tracker.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs....
    opened by GY56723-debug 1
  • About TSDF Fusion

    About TSDF Fusion

    Regarding tsdf map fusion, there is no detailed explanation in the paper. I would like to ask a few questions, which I would be grateful for.

    1. When building a map in real time, how is the size of TSDF map initialized?
    2. Traditional TSDF algorithm can't deal with the occluded surface. Is there any special treatment for TSDF fusion in Tandem? What's the difference between it and conventional TSDF fusion? Thank you very much for your guidance！
    opened by bscbje 0
Owner
TUM Computer Vision Group
TUM Computer Vision Group
GitHub
ORB-SLAM3 is the first real-time SLAM library able to perform Visual, Visual-Inertial and Multi-Map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models.

Just to test for my research, and I add coordinate transformation to evaluate the ORB_SLAM3. Only applied in research, and respect the authors' all work.

B.X.W 5 Jul 11, 2022
OrcVIO Stereo and object mapping

About Object residual constrained Visual-Inertial Odometry (OrcVIO) is a visual-inertial odometry pipeline, which is tightly coupled with tracking and

Sean 13 Oct 27, 2022
A 3D DNN-based Metric Semantic Dense Mapping pipeline and a Visual Inertial SLAM system

MSDM-SLAM This repository represnets a 3D DNN-based Metric Semantic Dense Mapping pipeline and a Visual Inertial SLAM system that can be run on a grou

ITMO Biomechatronics and Energy Efficient Robotics Laboratory 11 Jul 23, 2022
Cartographer is a system that provides real-time simultaneous localization and mapping (SLAM) in 2D and 3D across multiple platforms and sensor configurations.

Cartographer Purpose Cartographer is a system that provides real-time simultaneous localization and mapping (SLAM) in 2D and 3D across multiple platfo

Cartographer 6.3k Jan 4, 2023
Real time eye tracking for embedded and mobile devices.

drishti Real time eye tracking for embedded and mobile devices in C++11. NEWS (2018/08/10) Native iOS, Android, and "desktop" variants of the real-tim

null 356 Dec 14, 2022
R3live - A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package

R3LIVE A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package News [Dec 31, 2021] Release of cod

HKU-Mars-Lab 1.3k Jan 4, 2023
Dense Depth Estimation from Multiple 360-degree Images Using Virtual Depth

Dense Depth Estimation from Multiple 360-degree Images Using Virtual Depth [Project] [Paper] [arXiv] This is the official code of our APIN 2022 paper

null 8 Nov 7, 2022
OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

Build Type Linux MacOS Windows Build Status OpenPose has represented the first real-time multi-person system to jointly detect human body, hand, facia

null 25.6k Dec 29, 2022
A multi object tracking Library Based on tensorrt

YoloV5_JDE_TensorRT_for_Track Introduction A multi object detect and track Library Based on tensorrt 一个基于TensorRT的多目标检测和跟踪融合算法库，可以同时支持行人的多目标检测和跟踪，当然也可

zwg_cv 48 Nov 25, 2022
Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models

Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine DSSTNE (pronounced "Destiny") is an open source software library for training and deploying

Amazon Archives 4.4k Dec 30, 2022
Colibri core is an NLP tool as well as a C++ and Python library for working with basic linguistic constructions such as n-grams and skipgrams (i.e patterns with one or more gaps, either of fixed or dynamic size) in a quick and memory-efficient way. At the core is the tool ``colibri-patternmodeller`` whi ch allows you to build, view, manipulate and query pattern models.

Colibri Core by Maarten van Gompel, [email protected], Radboud University Nijmegen Licensed under GPLv3 (See http://www.gnu.org/licenses/gpl-3.0.html

Maarten van Gompel 122 Nov 17, 2022
Anomaly Detection on Dynamic (time-evolving) Graphs in Real-time and Streaming manner

Anomaly Detection on Dynamic (time-evolving) Graphs in Real-time and Streaming manner. Detecting intrusions (DoS and DDoS attacks), frauds, fake rating anomalies.

Stream-AD 696 Dec 18, 2022
Faiss is a library for efficient similarity search and clustering of dense vectors.

Faiss is a library for efficient similarity search and clustering of dense vectors. It contains algorithms that search in sets of vectors of any size, up to ones that possibly do not fit in RAM. It also contains supporting code for evaluation and parameter tuning. Faiss is written in C++ with complete wrappers for Python/numpy.

Facebook Research 18.7k Jan 1, 2023
Real time monaural source separation base on fully convolutional neural network operates on Time-frequency domain.

Real time monaural source separation base on fully convolutional neural network operates on Time-frequency domain.

James Fung 111 Jan 9, 2023
Implementations of Multiple View Geometry in Computer Vision and some extended algorithms.

MVGPlus Implementations of Multiple View Geometry in Computer Vision and some extended algorithms. Implementations Template-based RANSAC 2D Line estim

Chenyu 6 Apr 7, 2022
open Multiple View Geometry library. Basis for 3D computer vision and Structure from Motion.

OpenMVG (open Multiple View Geometry) License Documentation Continuous Integration (Linux/MacOs/Windows) Build Code Quality Chat Wiki local/docker bui

openMVG 4.6k Jan 8, 2023
General broad-phase collision detection framework using BVH and BVTT front tracking.

This is the collision detection package by littlemine (Xinlei Wang). Configuration Instructions This project is developed using Visual Studio 2015 and

Xinlei Wang 47 Sep 9, 2022
A project demonstrating how to train your own gesture recognition deep learning pipeline. We start with a pre-trained detection model, repurpose it for hand detection using Transfer Learning Toolkit 3.0, and use it together with the purpose-built gesture recognition model. Once trained, we deploy this model on NVIDIA® Jetson™ using Deepstream SDK.

Using NVIDIA Pre-trained Models and Transfer Learning Toolkit 3.0 to Create Gesture-based Interactions with a Robot In this project, we demonstrate ho

NVIDIA AI IOT 31 Dec 15, 2022
NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Real-ESRGAN ncnn Vulkan This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many

Xintao 602 Jan 6, 2023