6D - Pose Annotation Tool (6D-PAT) - is a tool that allows the user to load a set of images and also a set of 3D models and annotate where in the 2D image the 3D object ist placed.

Related tags

Miscellaneous 6d-pat
Overview

The 6D-PAT logo

6D - Pose Annotation Tool (6D-PAT)

GitHub release (latest by date) Build Coverage Status CodeFactor contributions welcome GitHub license

For detiled explanations checkout the WikiPage.

What is it?

With 6D-PAT you can create 6D annotations on images for 6D pose estimation, i.e. annotate 2D images with the 3D rotation and 3D translation of 3D models.

How does it work?

The program allows you to select a folder and view the images contained in it in a gallery. Selecting one of the images will display it at a larger scale to create new 6D pose annotations. The 3D models for those annotations are displayed in a second gallery which also loads the models from a specified folder. In the 3D viewer of the program, you can inspect a selected 3D model, rotate it and use it to create a new pose annotation.

Imgur

Objects and images are from the T-Less Dataset.

Getting the program

Running the AppImage

Note: The AppImage is built on Ubuntu 20.04 (the latest version) and thus requries you to have Ubuntu 20.04. You could try a virtual machine if you Ubuntu version doesn't match or build the program yourself.

You can download the latest AppImage from the releases page which contains everything the program needs to run and should work on the latest Ubuntu out of the box.

Run the Docker image

Enable X-server display for Docker:

xhost +local:root

Run the Docker image (command changed! omit /6DPAT at the end, this is now in the entrypoint):

docker run -ti --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix -v /dev/dri/card0:/dev/dri/card0 florianblume/6dpat

Check out the getting the program wiki page for more details.

Build from source

Requirements

OpenGL Qt OpenCV Python Pybind11
Version >=3.1 >=5.14 >=4.5 ==3.8 ==2.6.2

For OpenGL, Qt and OpenCV these are the minimum versions you need to have installed. Python needs to be exactly 3.8 (C++ interface changes from version to version in Python) and I'm not sure about Pybind11 that's why I'm assuming you need exactly this version. You can do so this way for example (if you don't want to manually install Qt and build OpenCV):

sudo add-apt-repository -y ppa:beineri/opt-qt-5.14.2-focal
sudo apt-get update -qq
sudo apt-get -y install qt514-meta-minimal qt5143d qt514gamepad python3 python3-dev python3-pybind11
sudo apt-get -y install libopencv-dev

Then open the project's main 6d-pat.pro file in QtCreator and build the project. Everything should compile successfully. If not: Feel free to open an issue and I'll try to help you.

Setting up the program the first time

Check out the program setup wiki page to see in detail how to set up the program.

Recovering poses

To start recovering poses, follow these steps:

  1. Select an image
  2. Select the corresponding object model
  3. Rotate the object model to a similar position as visible in the image
  4. Click on the image on a characteristic point of the object
  5. Click the same point on the 3D model - the program will show the number of click points at the bottom left
  6. Repeat setps 4 - 5 until at least 6 correspondences were created - more correspondences help to make the pose more accurate
  7. Click the "Create" button at the bottom of the pose editor
  8. You should see the recovered pose on the image
  9. You can refine it using the number fields or by dragging or rotating it directly with the mouse after selecting the 3D model
  10. After pose refinement, don't forget to press "Save"

More steps and details are on the wiki page.

Issues
  • Enabling 3D acceleration crashes program

    Enabling 3D acceleration crashes program

    I have installed Virtualbox 6 with ubuntu 20.04 VM. When I run the App image without 3D acceleration, the app works but I get very sluggish performance, always running at 1FPS. So I tried enabling 3D acceleration to improve performance but when I run the AppImage, the program shows the 3D PAT initial image but then it crashes.

    The problem remains with and without Virtualbox guest additions installed.

    opened by JoaoGuimaraes98 15
  • Docker not working on Ubuntu 16.04 LTS

    Docker not working on Ubuntu 16.04 LTS

    Hi, first of all I would just like to say that I am very new to Ubuntu and Docker so bear with me.

    I am running Ubuntu 16.04 LTS and QT 5.5.1 and I installed docker and when I run the image the application Starts it doesn't show graphics and I get this error multiple times in the command line:

    [unknown] (warning): composeAndFlush: makeCurrent() failed [unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed [unknown] (warning): bool Qt3DRender::Render::GraphicsContext::makeCurrent(QSurface*) makeCurrent failed

    Do you know how I can fix this problem? Also can you tell me if the program supports 3D models in the .ply format?

    Cheers and thanks for the help.

    opened by JoaoGuimaraes98 15
  • AppImage still not working

    AppImage still not working

    Hi florian, i'm sorry to be annoying. It's not a big deal because i installed 6dpat building it from source 2 months ago. Now that i need it on another computer i decided to retry with the AppImage, because i read on the issue you closed that the problem was solved.

    Originally posted by @florianblume in https://github.com/florianblume/6d-pat/issues/121#issuecomment-874825607

    When i run the appImage on my Ubuntu 20.04 the result is:

    "./6DPAT-x86_64.AppImage: error while loading shared libraries: libopencv_calib3d.so.4.3: cannot open shared object file: No such file or directory"

    I tried with manual installation of libopencv but nothing changed. Thanks for your work.

    opened by tobi1modna 11
  • Always

    Always "...clicked at: (0, 0, 0)."

    Hello,

    I successfully loaded my images and 3d model (i used .ply). But when I try to click on the 3D model (in the GUI) it always says I clicked at location (0,0,0), which is obviously not the case. What am I doing wrong?

    I have set up my camera parametrics correctly as well.

    bug 
    opened by Saafke 10
  • Build fail

    Build fail

    Hi,

    When I try to build the project in QtCreator or from cmd line it fails.

    In file included from ../../6d-pat-2.0/src/view/poseviewer/poseviewer.hpp:7, from ../../6d-pat-2.0/src/view/mainwindow.hpp:7, from ../../6d-pat-2.0/src/controller/poseseditingcontroller.hpp:8, from ../../6d-pat-2.0/src/controller/poseseditingcontroller.cpp:1: ../../6d-pat-2.0/src/view/poseviewer/poseviewer3dwidget.hpp:4:10: fatal error: qt3dwidget.h: No such file or directory 4 | #include "qt3dwidget.h" | ^~~~~~~~~~~~~~

    The build cannot find qt3dwidget.h

    opened by AnasIbrahim 8
  • libGL error on Installation

    libGL error on Installation

    Hi Florian,

    Great looking software, thanks for sharing. Just trying to use the docker image you have provided. I have tried using this with an Ubuntu 16.04 and 18.04 machine with proprietary Nvidia drivers installed (415 & 435) and can get to the splash screen before a segmentation fault.

    I am always met by this error:

    docker run -ti --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix florianblume/6dpat [unknown] (warning): QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root' libGL error: No matching fbConfigs or visuals found libGL error: failed to load driver: swrast [unknown] (warning): QFileSystemWatcher::removePath: path is empty [unknown] (warning): QFileSystemWatcher::removePath: path is empty [unknown] (warning): QFileSystemWatcher::removePath: path is empty [unknown] (warning): QOpenGLFunctions created with non-current context Segmentation fault (core dumped)

    I have tried hard to debug this error by going through mainly Steam-related threads on Google with this problem. However, reinstalling Nvidia drivers, looking for problematic mesa drivers, etc. makes no difference. I can run glxgears on my system proving OpenGL is working and installed nvidia-docker, however this makes no difference.

    I have also tried on a Jetson Nano device with preinstalled drivers, docker, etc. but am met by a "exec user process caused exec format error" so have not pursued this much further.

    Any assistance would be great!

    Kind regards, Tom.

    P.S. I have also tried the docker image for just installing the environment but am met by the same issue.

    opened by tom-robo 4
  • Make offscreen rendering of 3D models thread-safe

    Make offscreen rendering of 3D models thread-safe

    Currently, the program sometimes crashes because the offscreen renderer returns and tries to insert the newly rendered image while the program is already shutting down. To avoid this, a possible solution is to store the offscreen renderer threads and cancel them in the GalleryObjectModelModel. Maybe some blocking instructions could help, too.

    bug 
    opened by florianblume 4
  • AppImage:unable to display enlarged image

    AppImage:unable to display enlarged image

    Hello, I have run the appimage(v2.4.2) on ubuntu 20.04. but the preview image's displaying box is not in right size and unable to display enlarged images. I impose my dataset configuration file(Default JSON file) may have some problem. Would you enjoy to share your dataset configuration as reference to help me figure out the problem?
    6D

    bug 
    opened by maxmin2 3
  • Bugfix for the persist of poses

    Bugfix for the persist of poses

    Hi, i think i found a bug: after some days using the tool i wasn't able to save the poses. I press save, bot nothing was written in the gt.json file. I'm using the program in the "Default Json" mode. The gt.json file path is correctly set in the settings.

    Everytime jus the first pose was saved, but never more than one.

    So, i checked the file https://github.com/florianblume/6d-pat/blob/3b42914d0a347d24d87de25e27d562ccaaf8ad15/src/model/jsonloadandstorestrategy.cpp and changed at line 43.

    from: if (jsonDocument.isNull()) { to: if (!jsonDocument.isNull()) {

    This corrected the problem, i re-built the program and now i can save more than one poses in the json file. Reading your comment at line 115, in the ELSE: // TODO JsonDocument is null, ie an error occured i thought that you wanted to write "if jsonDocument is not null, then ...", and not "if null, then ..."

    opened by tobi1modna 3
  • Exporting images/annotations

    Exporting images/annotations

    Hi Florian, is there a way to export the annotations about the pose of the model over the 2D image and to save images with the "blue overlapped model" on them? If not, do you think this feature would be easily extendable by myself?

    opened by tobi1modna 3
  • Name poses by adjective-animal scheme

    Name poses by adjective-animal scheme

    Instead of using a UUID, name poses with something like "scary-donky", etc. This makes poses easily identifiable and with enough combinations serves as a unique identifier.

    opened by florianblume 3
  • The annotation result image didn't show

    The annotation result image didn't show

    I tried to annotate the image, but the result didn't show in the interface. The box just remained black, so I can't check my pose annotation. Software edition (6D-PAT): 2.5 Ubuntu version: 20.04 Screenshot from 2022-06-29 20-16-09

    opened by sean6585 20
  • If there is only one object model or image, on reloading views it's not selectable

    If there is only one object model or image, on reloading views it's not selectable

    Currently, when the user selects an object model or image and there is only one in the gallery and then the user reloads the views they cannot select the item since it has been selected already. I need to reset the selection when reloading the views.

    bug 
    opened by florianblume 0
  • Make model managers return values not pointers

    Make model managers return values not pointers

    The memory overhead from returning values instead of pointers will be minimal since there do not exist many copies of the data throughout the program. Right now, many issues arise from returning shared pointers.

    But before implementing this, write tests for the most crucial parts like the model managers.

    enhancement 
    opened by florianblume 0
Releases(v2.5.2)
Owner
Florian Blume
PhD student at TU Berlin.
Florian Blume
Phan Sang 4 Jun 21, 2022
Wayab(wayland animated background) allows user to set animated background on Linux(wayland).

wayab wayab(wayland animated background) allows user to set animated background on Linux(wayland). Features low CPU costs. Unlike oguri which change t

Yongsheng Xu 9 Jun 15, 2022
imGuIZMO.quat is a ImGui widget: like a trackball it provides a way to rotate models, lights, or objects with mouse, and graphically visualize their position in space, also around any single axis (Shift/Ctrl/Alt/Super)

imGuIZMO.quat v3.0 imGuIZMO.quat is a ImGui widget: like a trackball it provides a way to rotate models, lights, or objects with mouse, and graphicall

Michele Morrone 254 Jun 12, 2022
Xournal++ is a handwriting notetaking software with PDF annotation support

Xournal++ is a handwriting notetaking software with PDF annotation support. Written in C++ with GTK3, supporting Linux (e.g. Ubuntu, Debian, Arch, SUSE), macOS and Windows 10. Supports pen input from devices such as Wacom Tablets.

Xournalpp 6.7k Jun 26, 2022
Tsdf-plusplus - TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction TSDF++ is a novel multi-object TSDF formulation that can encode mult

ETHZ ASL 111 Jun 21, 2022
Allows for multiple SwitchBot buttons and curtains to be controlled via MQTT sent to ESP32. ESP32 will send BLE commands to switchbots and return MQTT responses to the broker. Also supports Meter/Temp Sensor

SwitchBot-MQTT-BLE-ESP32 Switchbot local control using ESP32. no switchbot hub used/required. works with any smarthub that supports MQTT https://githu

null 244 Jun 29, 2022
Blazingly fast multi-object tracker. Works on 1 (x, y) point per object.

Norfair++ This is a C++ implementation of Techainer's Norfair, which originates from Norfair, a library for real-time 2D object tracking. Its function

Tô Đức (Watson) 6 Jan 26, 2022
A sketch that not only parses NMEA sentences, but also allows sending UBX commands and decrypt answers from the ublox module

RAK4631-ublox-Commander This is a tokenizer and parser for raw NMEA sentences. This is not intended (yet anyway) for production, but as an exercice in

Kongduino 2 May 29, 2022
Second life for famous JPEGView - fast and tiny viewer/editor for JPEG, BMP, PNG, WEBP, TGA, GIF and TIFF images with a minimalist GUI and base image processing.

JPEGView-Image-Viewer-and-Editor Updated Dec 07 2021. Version 1.1.1.0 has been released. Download link1, link2 added. Second life for famous JPEGView

Ann Hatt 20 Jun 9, 2022
ToPS is an objected-oriented framework implemented using C++ that facilitates the integration of probabilistic models for sequences over a user defined alphabet

ToPS is an objected-oriented framework implemented using C++ that facilitates the integration of probabilistic models for sequences over a user defined alphabet

Andre Yoshiaki Kashiwabara 35 Mar 6, 2022
anthemtotheego 330 Jul 1, 2022
Performance Evaluation of a Parallel Image Enhancement Technique for Dark Images on Multithreaded CPU and GPU Architectures

Performance Evaluation of a Parallel Image Enhancement Technique for Dark Images on Multithreaded CPU and GPU Architectures Image processing is a rese

Batuhan Hangün 5 Nov 4, 2021
🖼️ A writeable in-memory Image JSI Host Object

??️ react-native-jsi-image ??️ This library is work in progress! ??️ A writeable in-memory Image JSI Host Object. JSI-Image is a modern library that p

Marc Rousavy 240 Jun 30, 2022
Icopack - A simple command line tool to create multi-frame ICO files from PNG source images

Optidash is a modern, AI-powered image optimization and processing API. We will drastically speed-up your websites and save you money on bandwidth and

Optidash AI 59 Jun 15, 2022
I modified the colmap,when it reconstructs from known pose ,only let it optimize rotation ,fixing position!

Mapping-base-lidar-pose-or-vslam-pose I simply modified the colmap,when it reconstructs from known pose ,only let it optimize rotation ,fixing positio

李城(大橙子) 19 Jun 20, 2022
QPEP (Quadratic Pose Estimation Problems) Enhanced VINS-Mono SLAM System

VINS-Mono-QPEP The QPEP (Quadratic Pose Estimation Problems) Enhanced VINS-Mono

Jin Wu 26 Jun 28, 2022
The ROS version of ICP Mapping with QPEP Solver (Quadratic Pose Estimation Problems)

The ROS version of ICP Mapping with QPEP Solver (Quadratic Pose Estimation Problems) The project is based on https://github.com/ethz-asl/ethzasl_icp_m

Jin Wu 10 Feb 25, 2022
MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files supporting textures, skeletons, and all provided vertex data.

MDE is a model extraction tool that converts Destiny 2 dynamic models into fbx files. A dynamic model is one that is animated or is spawned in during the game.

Montague 31 Jun 20, 2022
Load and execute COFF files and Cobalt Strike BOFs in-memory

COFFLoader2 This repo contains the source code of a Common Object File Format (COFF) loader, which is a rewrite of the research and implementation don

Yasser 109 Jun 27, 2022