Android MoveNet pose estimation by ncnn

Overview

ncnn_Android_MoveNet

Android MoveNet single human pose estimation by ncnn

this project is a ncnn Android demo for MoveNet, it depends on ncnn library and opencv.
https://github.com/Tencent/ncnn
https://github.com/nihui/opencv-mobile

model support:

1.movenet-singlepose-lightningv4(from tfhub)
2.movenet-singlepose-thunderv4(from tfhub)

how to build and run

step1

https://github.com/Tencent/ncnn/releases

  • Download ncnn-YYYYMMDD-android-vulkan.zip or build ncnn for android yourself
  • Extract ncnn-YYYYMMDD-android-vulkan.zip into app/src/main/jni and change the ncnn_DIR path to yours in app/src/main/jni/CMakeLists.txt

step2

https://github.com/nihui/opencv-mobile

  • Download opencv-mobile-XYZ-android.zip
  • Extract opencv-mobile-XYZ-android.zip into app/src/main/jni and change the OpenCV_DIR path to yours in app/src/main/jni/CMakeLists.txt

step3

  • Open this project with Android Studio, build it and enjoy!

result

reference:

https://github.com/nihui/ncnn-android-nanodet
https://tfhub.dev/google/movenet/singlepose/lightning/4
https://tfhub.dev/google/movenet/singlepose/thunder/4

Comments
  • The speed might not fast as the tflite version?

    The speed might not fast as the tflite version?

    Hi there, I used the lightning model and the after-treatment from this project and find the fps is about 20 and up to 25 which is not fast as the original tflite version. I'm not sure the reason, there's my doubt:

    1. the after-treatment cause more computing resources?
    2. there are some opt. in ncnn can be opened, such as vulkan?
    3. the model in ncnn is natively slower than tflite cuz the tflite model is also optimized? Could you give some enlightenment about these? Thx
    opened by FreedoomG 2
  • gpu cpu两种模式没差别

    gpu cpu两种模式没差别

    cpu占用基本一眼,帧率基本一样 设备 amlogic 311D PID USER PR NI VIRT RES SHR S[%CPU] %MEM TIME+ ARGS 20344 u0_a55 10 -10 1.2G 207M 104M S 284 10.3 32:20.16 com.tencent.ncnnbodypose

    20344 u0_a55 10 -10 1.2G 209M 104M S 272 10.4 33:09.35 com.tencent.ncnnbodypose

    opened by lovehuanhuan 0
  • issue about the output of the model

    issue about the output of the model

    hi I found that there are 4 outputs of lighting.param and thunder.param which are regress, offset, heatmap and center. could you give some advice of how to use these outputs to generate the keypoints? cuz I cannot find the code here. Thx!

    opened by FreedoomG 0
  • @feigechuanshu can you make a face recognition app using NCNN? maybe MobileFaceNet?

    @feigechuanshu can you make a face recognition app using NCNN? maybe MobileFaceNet?

    could you bring something like this to android ? https://github.com/honghuCode/mobileFacenet-ncnn

    for real-time face recognition on live video. With re-training capabilities, in which user can retrain any given face name with additional data.

    opened by GeorvityLabs 0
  • 横屏时陀螺仪不起作用导致识别不准

    横屏时陀螺仪不起作用导致识别不准

    机型:红米K30 NDK版本:android-ndk-r21e camke版本:3.10.2-win64-x64

    ALooper_pollAll的返回值一直都是-3,进不去下面的陀螺仪事件里去,导致横屏时识别效果很差,因为图像没旋转的缘故,不知道为什么。

    image

    但是这里无视这个错误继续进入那个if分支的话好像也没啥问题,横屏时陀螺仪也能正常工作。

    opened by Yamilemon 0
  • How to hide the current camera screen?

    How to hide the current camera screen?

    Hello,

    I want to implement the title function, and pass the value to the U3D painting after obtaining the detection result. How can I hide my current display screen?

    opened by jia0511 1
  • model convert

    model convert

    thank you for this nice work.Now I want to convert the tensorflow hub's official movenet model to ncnn. My convert steps as follows: 1、convert pb to onnx; 2、optimize and simplifier onnx model; 3、convert onnx to ncnn; However there some ops ncnn dont support. Error msg as follows: Cast not supported yet!

    to=1

    Unsupported split axis ! ArgMax not supported yet!

    axis=1

    keepdims=0

    Cast not supported yet!

    to=6

    Cast not supported yet!

    to=1

    Cast not supported yet!

    to=7

    GatherND not supported yet! Cast not supported yet!

    to=1

    ArgMax not supported yet!

    axis=1

    keepdims=0

    Cast not supported yet!

    to=6

    Cast not supported yet!

    to=1

    Cast not supported yet!

    to=7

    GatherND not supported yet! Unsupported unsqueeze axes ! Unsupported unsqueeze axes ! Cast not supported yet!

    to=7

    GatherND not supported yet! GatherND not supported yet! Unsupported unsqueeze axes ! Unsupported unsqueeze axes ! Cast not supported yet!

    to=1

    Unsupported unsqueeze axes ! It will be very nice if you can tell me how do you solve this problem, thank you!

    opened by ccslience 1
  • Convert .pt to .bin and .param format

    Convert .pt to .bin and .param format

    I trained my model yolov5 for object detection and now i want to deploy it on android app. Before I tried covert to .tflite format but slowly. I saw in assets folder the format you used is .bin and .param format. What code did you use to convert my model to format .bin and .param ?

    opened by Houangnt 0
Owner
FeiGeChuanShu
FeiGeChuanShu
Pose-tensorflow - Human Pose estimation with TensorFlow framework

Human Pose Estimation with TensorFlow Here you can find the implementation of the Human Body Pose Estimation algorithm, presented in the DeeperCut and

Eldar Insafutdinov 1.1k Dec 29, 2022
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

null 272 Jan 3, 2023
This is a sample ncnn android project, it depends on ncnn library and opencv

This is a sample ncnn android project, it depends on ncnn library and opencv

null 248 Jan 6, 2023
Movenet cpp deploy; model transformed from tensorflow

MoveNet-PaddleLite Adapted from PaddleDetection; Movenet cpp deploy based on PaddleLite; Movenet model transformed from tensorflow; 简介 Movenet是近年的优秀开源

null 11 Dec 27, 2022
Fast and robust certifiable relative pose estimation

Fast and Robust Relative Pose Estimation for Calibrated Cameras This repository contains the code for the relative pose estimation between two central

null 42 Dec 6, 2022
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV Workshop @ CVPR 2021.

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022
CUDA-accelerated Apriltag detection and pose estimation.

Isaac ROS Apriltag Overview This ROS2 node uses the NVIDIA GPU-accelerated AprilTags library to detect AprilTags in images and publishes their poses,

NVIDIA Isaac ROS 46 Dec 26, 2022
Real-Time Neural 3D Hand Pose Estimation from an Event Stream [ICCV 2021]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream Project Page Index TRAIN.md -- how to train the model from scratch EVAL_REAL

null 23 Nov 7, 2022
Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation This is the implementation of the approach describ

Taosha Fan 47 Nov 15, 2022
HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)

HybridPose: 6D Object Pose Estimation under Hybrid Representations This repository contains authors' implementation of HybridPose: 6D Object Pose Esti

SONG, Chen 358 Nov 22, 2022
Simple inference deep head pose ncnn version

ncnn-deep-head-pose Simple implement inference deep head pose ncnn version with high performance and optimized resource. This project based on deep-he

Đỗ Công Minh 13 Dec 16, 2022
GFPGAN-ncnn - a naive NCNN implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration

GFPGAN-ncnn a naive ncnn implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration model support: 1.GFPGANClean

FeiGeChuanShu 42 Dec 10, 2022
A lightweight 2D Pose model can be deployed on Linux/Window/Android, supports CPU/GPU inference acceleration, and can be detected in real time on ordinary mobile phones.

A lightweight 2D Pose model can be deployed on Linux/Window/Android, supports CPU/GPU inference acceleration, and can be detected in real time on ordinary mobile phones.

JinquanPan 58 Jan 3, 2023
pose demo on android mobile based on PaddleDetection

pose_demo_android pose demo on android mobile based on PaddleDetection 本工程Android部分基于Paddle-Lite-Demo修改。 算法模型基于PaddleDetection的PP-TinyPose. 如欲获取更多详情,请

null 52 Dec 9, 2022
Android face detect and segmentation by ncnn

Android face detect and segmentation by ncnn

null 118 Dec 29, 2022
Android hair/human segmentation demo by ncnn

ncnn_Android_human Android hair/human segmentation demo by ncnn PS:performance maybe poor.it's just a demo:) Reference: 1.https://github.com/Tencent/n

null 23 May 23, 2022
Android yolox hand detect by ncnn

The yolox hand detection This is a sample ncnn android project, it depends on ncnn library and opencv https://github.com/Tencent/ncnn https://github.c

FeiGeChuanShu 14 Sep 7, 2022
PiDiNet running in Android by ncnn

PiDiNet-NCNN 使用NCNN在安卓上实现PiDiNet这个边缘检测网络。 什么是PiDiNet?指南针:https://github.com/zhuoinoulu/pidinet 文件构成 ncnn-android-nanodet是基于nihui的nanodet项目魔改的,被我改成了运行P

WuJinxuan 9 Dec 13, 2022
monodepth running in Android by ncnn

monodepth-NCNN 将wavelet-monodepth的模型搬运到NCNN上,工程里面给了安卓的工程以及以及生成好的app安装包 wavelet-monodepth wavelet-monodepth:RGB图像的深度估计,wavelet顾名思义,就使用了小波变换的,官方的工程在这:ht

WuJinxuan 14 Aug 13, 2022