[CVPR 2021] NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning

Overview

NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning

Project Page | Paper | Supplemental material #1 | Supplemental material #2 | Presentation Video

Hyunho Ha ([email protected]), Joo Ho Lee ([email protected]), Andreas Meuleman ([email protected]) and Min H. Kim ([email protected])

Institute: KAIST Visual Computing Laboratory

If you use our code for your academic work, please cite our paper:

@InProceedings{Ha_2021_CVPR,
	author = {Hyunho Ha and Joo Ho Lee and Andreas Meuleman and Min H. Kim},
	title = {NormalFusion: Real-Time Acquisition of Surface Normals for High-Resolution RGB-D Scanning},
	booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
	month = {June},
	year = {2021}
}

Installation

Our implementation is based on the voxel hashing (https://github.com/niessner/VoxelHashing) and TextureFusion repository (https://github.com/KAIST-VCLAB/texturefusion).

To run our code, first obtain the entire source codes from voxel hashing repository, including the Visual Studio project file. Then, in VoxelHashing/DepthSensingCUDA/, replace the folders Source/ and Shaders/ as well as the configuration files zParameters*.txt by the content of our repository. Therefore, our source code inherits the dependency of the Voxel Hashing project as follows.

Our work requires:

Our code has been developed with Microsoft Visual Studio 2013 (VC++ 12) and Windows 10 (10.0.19041, build 19041) on a machine equipped with Intel i9-10920X (RAM: 64GB), NVIDIA TITAN RTX (RAM: 24GB). The main function is in normalFusion_main.cpp.

Data

We provide the "fountain" dataset (originally created by Zhou and Koltun) compatible with our implementation (link: http://vclab.kaist.ac.kr/cvpr2020p1/fountain_all.zip).

Usage

Our program reads parameters from three files and you can change the program setting by changing them.

  • zParametersDefault.txt

  • zParametersTrackingDefault.txt

  • zParametersWarpingDefault.txt

  • zParametersEnhancementDefault.txt

You can run our program with the provided fountain dataset.

Please set s_sensorIdx as 9 and s_binaryDumpSensorFile[0] as the fountain folder in zParametersDefault.txt.

Our program produces mesh with two textures (diffuse albedo and normal). If you want to further enhance mesh using normal texture, please refer to the paper: "Efficiently Combining Positions and Normals for Precise 3D Geometry", Nehab et al., ACM TOG, 2005.

License

Hyunho Ha, Joo Ho Lee, Andreas Meuleman, and Min H. Kim have developed this software and related documentation (the "Software"); confidential use in source form of the Software, without modification, is permitted provided that the following conditions are met:

Neither the name of the copyright holder nor the names of any contributors may be used to endorse or promote products derived from the Software without specific prior written permission.

The use of the software is for Non-Commercial Purposes only. As used in this Agreement, "Non-Commercial Purpose" means for the purpose of education or research in a non-commercial organisation only. "Non-Commercial Purpose" excludes, without limitation, any use of the Software for, as part of, or in any way in connection with a product (including software) or service which is sold, offered for sale, licensed, leased, published, loaned or rented. If you require a license for a use excluded by this agreement, please email [[email protected]].

Warranty: KAIST-VCLAB MAKES NO REPRESENTATIONS OR WARRANTIES ABOUT THE SUITABILITY OF THE SOFTWARE, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. KAIST-VCLAB SHALL NOT BE LIABLE FOR ANY DAMAGES SUFFERED BY LICENSEE AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.

Note that Our implementation inherits the original license of "Voxel Hashing" codes (CC BY-NC-SA 3.0).

Please refer to license.txt for more details.

Contact

If you have any questions, please feel free to contact us.

Hyunho Ha ([email protected])

Joo Ho Lee ([email protected])

Andreas Meuleman ([email protected])

Min H. Kim ([email protected])

Issues
Owner
KAIST VCLAB
KAIST Visual Computing Laboratory
KAIST VCLAB
ORB-SLAM3 is the first real-time SLAM library able to perform Visual, Visual-Inertial and Multi-Map SLAM with monocular, stereo and RGB-D cameras, using pin-hole and fisheye lens models.

Just to test for my research, and I add coordinate transformation to evaluate the ORB_SLAM3. Only applied in research, and respect the authors' all work.

B.X.W 5 Jul 11, 2022
R3live - A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package

R3LIVE A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package News [Dec 31, 2021] Release of cod

HKU-Mars-Lab 971 Aug 5, 2022
Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)

GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Mo

Abhinav Kumar 74 Jul 28, 2022
Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation"

SSM-VLN Code and Data for our CVPR 2021 paper "Structured Scene Memory for Vision-Language Navigation". Environment Installation Download Room-to-Room

hanqing 32 Jun 24, 2022
The official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averaging Approach

Graph Optimizer This repo contains the official implementation of our CVPR 2021 paper - Hybrid Rotation Averaging: A Fast and Robust Rotation Averagin

Chenyu 95 Aug 2, 2022
DeepI2P - Image-to-Point Cloud Registration via Deep Classification. CVPR 2021

#DeepI2P: Image-to-Point Cloud Registration via Deep Classification Summary Video PyTorch implementation for our CVPR 2021 paper DeepI2P. DeepI2P solv

Li Jiaxin 124 Jul 29, 2022
Real-time Skeletonization for Sketch-based Modeling (SMI:2021)

Real-time Skeletonization for Sketch-based Modeling (SMI:2021) Demo We provide an executable software under directory "demo_exe/". Tested Environment

null 231 Jul 13, 2022
Real-Time Neural 3D Hand Pose Estimation from an Event Stream [ICCV 2021]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream Project Page Index TRAIN.md -- how to train the model from scratch EVAL_REAL

null 19 Jun 30, 2022
🐸 Coqui STT is an open source Speech-to-Text toolkit which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers

Coqui STT ( ?? STT) is an open-source deep-learning toolkit for training and deploying speech-to-text models. ?? STT is battle tested in both producti

Coqui.ai 1.4k Jul 29, 2022
DeepRTS is a high-performance Real-TIme strategy game for Reinforcement Learning research written in C++

DeepRTS is a high-performance Real-TIme strategy game for Reinforcement Learning research. It is written in C++ for performance, but provides an python interface to better interface with machine-learning toolkits. Deep RTS can process the game with over 6 000 000 steps per second and 2 000 000 steps when rendering graphics. In comparison to other solutions, such as StarCraft, this is over 15 000% faster simulation time running on Intel i7-8700k with Nvidia RTX 2080 TI.

Centre for Artificial Intelligence Research (CAIR) 146 Jul 29, 2022
DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.

Project DeepSpeech DeepSpeech is an open-source Speech-To-Text engine, using a model trained by machine learning techniques based on Baidu's Deep Spee

Mozilla 20k Jul 29, 2022
HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)

HybridPose: 6D Object Pose Estimation under Hybrid Representations This repository contains authors' implementation of HybridPose: 6D Object Pose Esti

SONG, Chen 346 Jul 27, 2022
The repository contains our dataset and C++ implementation of the CVPR 2022 paper, Geometric Structure Preserving Warp for Natural Image Stitching.

Geometric Structure Preserving Warp for Natural Image Stitching This repository contains our dataset and C++ implementation of the CVPR 2022 paper, Ge

null 9 Jul 21, 2022
Real time monaural source separation base on fully convolutional neural network operates on Time-frequency domain.

Real time monaural source separation base on fully convolutional neural network operates on Time-frequency domain.

James Fung 100 Aug 5, 2022
Anomaly Detection on Dynamic (time-evolving) Graphs in Real-time and Streaming manner

Anomaly Detection on Dynamic (time-evolving) Graphs in Real-time and Streaming manner. Detecting intrusions (DoS and DDoS attacks), frauds, fake rating anomalies.

Stream-AD 681 Aug 3, 2022
Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, SRMD, RealSR, Anime4K, RIFE, CAIN, DAIN and ACNet.

Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, SRMD, RealSR, Anime4K, RIFE, CAIN, DAIN and ACNet.

Aaron Feng 7.3k Aug 7, 2022
Grand Theft Auto V + FidelityFx Super Resolution 1.0

Grand Theft Auto V + FidelityFx Super Resolution 1.0 This mod replaces original Grand Theft Auto V upscaler with FidelityFx Super Resolution 1.0 Origi

null 362 Jul 30, 2022
FidelityFX Super Resolution

FidelityFX Super Resolution 1.0 (FSR) Copyright (c) 2021 Advanced Micro Devices, Inc. All rights reserved. Permission is hereby granted, free of charg

GPUOpen Effects 1.8k Aug 2, 2022
Implementing Deep Convolutional Neural Network in C without External Libraries for YUV video Super-Resolution

DeepC: Implementing Deep Convolutional Neural Network in C without External Libraries for YUV video Super-Resolution This code uses FSRCNN algorithm t

Milad Abdollahzadeh 8 Jul 13, 2022