Example Qt application that demonstrates how to integrate Mediapipe

Overview

Mediapipe integration to Qt application example

Example on how to integrate mediapipe as a dynamic library into Qt applicaton on Linux.

Resulting application:

Mediapipe examples implemented are:

  • SelfieSegmentation
  • HandTracking
  • FaceDetection
  • FaceMesh
  • HairSegmentation
  • HolisticTracking
  • IrisTracking
  • ObjectDetection
  • PoseTracking

Guide

Guide assumes Ubuntu 18.04. Project was built using Qt 5.12, I don't know if it works on higher or lower versions.

  1. In order to build the mediapipe as .so, you need to clone my fork of mediapipe and checkout the linux-lib-example branch:

git clone https://github.com/asprecic/mediapipe.git && cd mediapipe && git checkout linux-lib-example

  1. You can build mediapipe using the provided Dockerfile or if you want to do it natively, follow mediapipe's guide on how to build it. My forked repo contains additional example under examples/desktop/libexample which you have to build like this:

bazel build --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/libexample:libexample.so

  1. You can find the built libexample.so under bazel-bin/mediapipe/examples/desktop/libexample/libexample.so. Copy libexample.so to EXAMPLE_LIB_PATH/lib. Sadly, header files don't get generated so you will have to manually copy example.h to EXAMPLE_LIB_PATH/include (location of example.h is mediapipe/examples/desktop/libexample/example.h). At this point, you have libexample.so and coresponding header file example.h which you can use to integrate into any application.
  2. Create user.pri and specify EXAMPLE_LIB_PATH
  3. Create mediapipe directory wherever the final binary of build is, and copy mediapipe/modules and mediapipe/models to it. In these folders, .tflite models can be found, without which example can't run. (poller will not return packets)
  4. Install dependencies apt install libopencv-core-dev libopencv-imgproc-dev libpulse-dev
  5. Build the Qt project and try the example.

Forked Mediapipe repo disables unused opencv dependencies as they bloat the dependencies (if all enabled, about 230 dependencies in total, including dumb dependencies like mysqlclient, sqlite and ffmpeg, which are not needed at all for these examples). If you need to work with videoio or use any other opencv dependencies, you can enable them for linking in mediapipe/third_party/opencv_linux.BUILD.

License

License is MIT, do whatever you want with this. If you found this example helpful, I would appreciate feedback that you found it useful, via github stars or anything.

Contributing

If you have any improvements to this example, please do suggest them and share them with the community. I accept PR's and any additions to this example (eg. Windows and MacOS integration)

Comments
  • How to load libexample.so at other application?

    How to load libexample.so at other application?

    Hi, @asprecic

    I want to compile libexample.so and use hand_pose at my own application not at QT. And how to call the API to do hand pose, do you have a demo code to share?

    Thanks in advance.

    opened by erwangccc 7
  • Where to put mediapipe/modules?

    Where to put mediapipe/modules?

    Hello! In README, you mentioned that:

    Create mediapipe directory wherever the final binary of build is, 
    and copy mediapipe/modules and mediapipe/models to it. 
    In these folders, .tflite models can be found, 
    without which example can't run. (poller will not return packets)
    

    Could you please explain where is "the final binary" more clearly?

    I haved tried to put these two folders in the folder of .so file and the Qt build folder, but both got wrong.

    "poselandmarkcpu_1__posedetectioncpu__inferencecalculator__poselandmarkcpu_1__posedetectioncpu__InferenceCalculator" failed: ; Can't find file: mediapipe/modules/pose_detection/pose_detection.tflite
    Calculator::Open() for node
    

    I am looking forward to your reply! Thanks in advance.

    opened by Billccx 2
  • Libopencv_core.so.3.2 not found

    Libopencv_core.so.3.2 not found

    ``/usr/bin/ld: warning: libopencv_core.so.3.2, needed by /home/rohit/mywork/mediapipe-qt-integration-example/example-lib/lib/libexample.so, not found (try using -rpath or -rpath-link) I am facing this error after running qmake MediapipeExample.pro and make. I have opencv installed (version 4.2). Seems like with apt install libopencv-core-dev opencv 4.2 gets installed but why is libopencv_core.so.3.2 is needed?

    opened by IamRo45 2
  • Failed to bazel build pre-built object_detection_offline_demo binary.

    Failed to bazel build pre-built object_detection_offline_demo binary.

    I notice there is a demo we can build by following the command at the end of dockerfile, but I can not find mediapipe/examples/desktop/demo in docker container. Can anybody tell me how to build the demo?

    thanks

    opened by Chris-He01 0
  • bulid libexample.dylib in macos bug run Undefined symbols for architecture x86_64:

    bulid libexample.dylib in macos bug run Undefined symbols for architecture x86_64:

    I'm use this

    cc_binary( name = "libexample.dylib", deps = [ ":bg_remover" ],

    linkshared = 1 ## Very important
    

    )

    build success libexample.dylib

    but use CMakeLists.txt run Undefined symbols for architecture x86_64: "StyleClass::qt_metacall(QMetaObject::Call, int, void**)", referenced from: vtable for QQmlPrivate::QQmlElement in graphs.cpp.o "StyleClass::qt_metacast(char const*)", referenced from: vtable for QQmlPrivate::QQmlElement in graphs.cpp.o "StyleClass::staticMetaObject", referenced from: int qmlRegisterType(char const*, int, int, char const*) in graphs.cpp.o QtPrivate::MetaObjectForType<StyleClass*, void>::value() in graphs.cpp.o QMetaTypeIdQObject<StyleClass*, 8>::qt_metatype_id() in graphs.cpp.o "ImageRenderer::qt_metacall(QMetaObject::Call, int, void**)", referenced from: vtable for QQmlPrivate::QQmlElement in main.cpp.o "ImageRenderer::qt_metacast(char const*)", referenced from: vtable for QQmlPrivate::QQmlElement in main.cpp.o "ImageRenderer::staticMetaObject", referenced from: int qmlRegisterType(char const*, int, int, char const*) in main.cpp.o QtPrivate::MetaObjectForType<ImageRenderer*, void>::value() in main.cpp.o QMetaTypeIdQObject<ImageRenderer*, 8>::qt_metatype_id() in main.cpp.o "MPIntegrationExample::fpsChanged(double)", referenced from: MPIntegrationExample::processImage(QImage const&)::$_0::operator()() const in mpintegrationexample.cpp.o "MPIntegrationExample::imageProcessed(QImage const&)", referenced from: MPIntegrationExample::processImage(QImage const&)::$_0::operator()() const in mpintegrationexample.cpp.o "StyleClass::metaObject() const", referenced from: vtable for QQmlPrivate::QQmlElement in graphs.cpp.o "ImageRenderer::metaObject() const", referenced from: vtable for QQmlPrivate::QQmlElement in main.cpp.o "typeinfo for StyleClass", referenced from: typeinfo for QQmlPrivate::QQmlElement in graphs.cpp.o "typeinfo for ImageRenderer", referenced from: typeinfo for QQmlPrivate::QQmlElement in main.cpp.o "vtable for StyleClass", referenced from: StyleClass::StyleClass() in graphs.cpp.o NOTE: a missing vtable usually means the first non-inline virtual member function has no definition. "vtable for ImageRenderer", referenced from: ImageRenderer::ImageRenderer(QQuickItem*) in imagerenderer.cpp.o ImageRenderer::~ImageRenderer() in main.cpp.o NOTE: a missing vtable usually means the first non-inline virtual member function has no definition. "vtable for MPIntegrationExample", referenced from: MPIntegrationExample::~MPIntegrationExample() in main.cpp.o MPIntegrationExample::MPIntegrationExample(QObject*) in mpintegrationexample.cpp.o NOTE: a missing vtable usually means the first non-inline virtual member function has no definition. ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: *** [mediapipe_qt_integration_example_master] Error 1 make[2]: *** [CMakeFiles/mediapipe_qt_integration_example_master.dir/all] Error 2 make[1]: *** [CMakeFiles/mediapipe_qt_integration_example_master.dir/rule] Error 2 make: *** [mediapipe_qt_integration_example_master] Error 2

    The QT run can't look the image

    opened by chfeizy 1
  • macos build for the same project

    macos build for the same project

    It's more of a bazel specific question. If I'm trying to generate .dylib for libexample, is changing the name in cc_binary sufficient? cc_binary( name = "libexample.dylib", deps = [ ":bg_remover" ], #linkopts = ["-Wl,-soname,libexample.dylib"], linkshared = 1 ## Very important ) and what should be the content of linkopts?

    opened by IamRo45 2
Owner
null
UE4 MediaPipe plugin

UE4 MediaPipe plugin Platforms: Win64 2D features: Face, Iris, Hands, Pose, Holistic 3D features: Face Mesh, World Pose Demo video: https://www.youtub

null 226 Dec 29, 2022
A mediapipe-hand demo infer by ncnn

The Android demo of Mediapipe Hand infer by ncnn Please enjoy the mediapipe hand demo on ncnn You can try this APK demo https://pan.baidu.com/s/1ArAMH

FeiGeChuanShu 48 Dec 22, 2022
The purpose of this project is to apply mediapipe to more AI chips.

1.About This Project Our Official Website: www.houmo.ai Who We Are: We are Houmo - A Great AI Company. We wish to change the world with unlimited comp

null 38 Nov 20, 2022
Native runtime package for MediaPipe.NET.

MediaPipe.NET.Runtime Native library package for MediaPipe.NET. This is the first half of the port of MediaPipeUnityPlugin, in order to use MediaPipe

Vignette 14 Oct 12, 2022
Implement yolov5 with Tensorrt C++ api, and integrate batchedNMSPlugin. A Python wrapper is also provided.

yolov5 Original codes from tensorrtx. I modified the yololayer and integrated batchedNMSPlugin. A yolov5s.wts is provided for fast demo. How to genera

weiwei zhou 46 Dec 6, 2022
This is an example of Artificial Intelligence (AI) calculations on a very cheap hardware.

This is an example of Artificial Intelligence (AI) calculations on a very cheap hardware.

null 2.9k Dec 31, 2022
This is a small example project, that showcases the possibility of using a surrogate model to estimate the drag coefficient of arbitrary triangles.

flowAroundTriangles This is a small example project, that showcases the possibility of using a surrogate model to estimate the drag coefficient of arb

null 6 Sep 16, 2022
A external memory allocator example for PyTorch.

Custom PyTorch Memory Management This is a external memory allocator example for PyTorch. The underlying memory allocator is CNMeM. Usage Compile with

Zilin Zhu 12 Aug 2, 2022
CUDA Custom Buffers and example blocks

gr-cuda CUDA Support for GNU Radio using the custom buffer changes introduced in GR 3.10. Custom buffers for CUDA-enabled hardware are provided that c

GNU Radio 5 Aug 17, 2022
Android example to get the rgb and disparity images from the OAK-D device connected to a phone.

depthai-android-jni-example (WIP) Android example to get the rgb and disparity images from the OAK-D device connected to a phone. DepthaiAndroidFast.m

Ibai Gorordo 30 Nov 27, 2022
An example of Pyston embedded in a C++ program.

EmbeddedPyston About This repository holds an example of the Pyston interpreter embedded within a C++ program. This repository is available as a GitHu

Molly 5 Apr 1, 2022
Example of using ultralytics YOLO V5 with OpenCV 4.5.4, C++ and Python

yolov5-opencv-cpp-python Example of performing inference with ultralytics YOLO V5, OpenCV 4.5.4 DNN, C++ and Python Looking for YOLO V4 OpenCV C++/Pyt

null 183 Jan 9, 2023
Example of using YOLO v4 with OpenCV, C++ and Python

yolov4-opencv-cpp-python Example of performing inference with Darknet YOLO V4, OpenCV 4.4.0 DNN, C++ and Python Looking for YOLO V5 OpenCV C++/Python

null 45 Jan 8, 2023
TensorVox is an application designed to enable user-friendly and lightweight neural speech synthesis in the desktop

TensorVox is an application designed to enable user-friendly and lightweight neural speech synthesis in the desktop, aimed at increasing accessibility to such technology.

null 143 Dec 15, 2022
EdgeKiller is a simple application that fully replaces Microsoft Edge with the Browser of choice.

EdgeKiller EdgeKiller is a simple application that fully replaces Microsoft Edge with the Browser of choice, while also intercepting all the microsoft

Jan Ochwat 2 Nov 30, 2021
Web Application to visualize TLE Data build for Space Apps 2021

This repository was created by Brian Donald, Bryan Pikaard, Zach Stence, and Andreas Wenzel for SpaceApps 2021. The Orbital Tracker uses TLE data to c

Brian D 5 Aug 28, 2022
Eclipse Deeplearning4J (DL4J) ecosystem is a set of projects intended to support all the needs of a JVM based deep learning application

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.

Eclipse Foundation 12.7k Dec 29, 2022
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

null 272 Jan 3, 2023
Example-application - Example out-of-tree application that is also a module

Zephyr Example Application This repository contains a Zephyr example application. The main purpose of this repository is to serve as a reference on ho

Zephyr Project 82 Dec 13, 2022
Example project which demonstrates various CMake features.

CMake example Example project which demonstrates various CMake features. CDash testing dashboard (probably empty but you can submit your test result t

Radovan Bast 142 Nov 4, 2022