Android hand detect and pose estimation by ncnn

Overview

ncnn_nanodet_hand
1.hand detect:用nanode-m训练了个hand detect模型,
2.hand pose:用CMU的数据集训练了个ghostnet作为backbone模仿pfld的handpose模型
3.推理:handpose.cpp单独检测pose,nanodet_hand.cpp单独检测手.参照该代码可以无痛集成到ncnn-android-nanodet中哦.
4.在nihui大佬的ncnn-android-nanodet项目基础上,去掉了其他模型只保留了检测手和手势的.
PS:由于数据集质量不理想,且自己水平有限所以实际效果可能较差,自己做着玩的.
image
Android上的效果可以看一下视频,总之ncnn和nanodet棒极了!
https://www.bilibili.com/video/BV1Uh411Q7Rq
Reference:
https://github.com/nihui/ncnn-android-nanodet(视频中的Android demo是基于该代码的)
https://github.com/Tencent/ncnn
https://github.com/RangiLyu/nanodet

Issues
  • error on device with single camera

    error on device with single camera

    Hi @FeiGeChuanShu ,

    Amazing project! It works fine on device with two cameras (front and back) , but fails on device with single camera (only back)

    my camera id is '1' , there is only a single camera in my device. so i set private int facing = 1;

    but, it is not working on the system because there is check for front camera that is causing an error.

    How can I fix this issue? Hope you can suggest a fix.

    This is my log output :

    2021-06-24 12:41:10.748 28982-28982/com.tencent.nanodetncnn E/NdkImageReader: AImageReader_getWindow 2021-06-24 12:41:11.040 28982-28982/com.tencent.nanodetncnn D/SurfaceView: checkSurfaceViewlLogProperty get invalid command 2021-06-24 12:41:11.054 28982-28982/com.tencent.nanodetncnn D/ncnn: loadModel 0x77c5483700 2021-06-24 12:41:11.057 28982-28982/com.tencent.nanodetncnn W/ncnn: load hand 2021-06-24 12:41:11.585 28982-28982/com.tencent.nanodetncnn D/ncnn: openCamera 1 2021-06-24 12:41:11.585 28982-28982/com.tencent.nanodetncnn W/NdkCamera: open 2021-06-24 12:41:11.588 28982-28982/com.tencent.nanodetncnn W/NdkCamera: open 0 2021-06-24 12:41:11.588 28982-28982/com.tencent.nanodetncnn E/ACameraManager: getCameraCharacteristics: Camera ID does not exist! 2021-06-24 12:41:11.588 28982-28982/com.tencent.nanodetncnn E/ACameraManager: openCamera: cannot get camera characteristics for camera . err -10001 2021-06-24 12:41:11.588 28982-28982/com.tencent.nanodetncnn E/NdkCameraDevice: ACameraDevice_createCaptureRequest: invalid argument! device 0x0 request 0x77bb99d238 2021-06-24 12:41:11.589 28982-28982/com.tencent.nanodetncnn A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x8 in tid 28982 (ent.nanodetncnn)

    opened by GeorvityLabs 1
  • how to get count of number of fingers open?

    how to get count of number of fingers open?

    @FeiGeChuanShu how do I get the count of the number of fingers open. Hope you could point out the edits needed to be made, to display the number of open fingers.

    opened by GeorvityLabs 0
Android MoveNet pose estimation by ncnn

ncnn_Android_MoveNet Android MoveNet single human pose estimation by ncnn this project is a ncnn Android demo for MoveNet, it depends on ncnn library

FeiGeChuanShu 74 Jul 20, 2022
Real-Time Neural 3D Hand Pose Estimation from an Event Stream [ICCV 2021]

EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream Project Page Index TRAIN.md -- how to train the model from scratch EVAL_REAL

null 19 Jun 30, 2022
Pose-tensorflow - Human Pose estimation with TensorFlow framework

Human Pose Estimation with TensorFlow Here you can find the implementation of the Human Body Pose Estimation algorithm, presented in the DeeperCut and

Eldar Insafutdinov 1.1k Jul 21, 2022
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

null 151 Aug 8, 2022
Android face detect and segmentation by ncnn

Android face detect and segmentation by ncnn

null 96 Jul 21, 2022
This is a sample ncnn android project, it depends on ncnn library and opencv

This is a sample ncnn android project, it depends on ncnn library and opencv

null 220 Jul 28, 2022
Fast and robust certifiable relative pose estimation

Fast and Robust Relative Pose Estimation for Calibrated Cameras This repository contains the code for the relative pose estimation between two central

null 40 Jul 23, 2022
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV Workshop @ CVPR 2021.

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 43 Jul 12, 2022
CUDA-accelerated Apriltag detection and pose estimation.

Isaac ROS Apriltag Overview This ROS2 node uses the NVIDIA GPU-accelerated AprilTags library to detect AprilTags in images and publishes their poses,

NVIDIA Isaac ROS 40 Aug 3, 2022
Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation

Reviatalizing Optimization for 3D Human Pose and Shape Estimation: A Sparse Constrained Formulation This is the implementation of the approach describ

Taosha Fan 45 Jul 27, 2022
A mediapipe-hand demo infer by ncnn

The Android demo of Mediapipe Hand infer by ncnn Please enjoy the mediapipe hand demo on ncnn You can try this APK demo https://pan.baidu.com/s/1ArAMH

FeiGeChuanShu 36 Aug 4, 2022
HybridPose: 6D Object Pose Estimation under Hybrid Representation (CVPR 2020)

HybridPose: 6D Object Pose Estimation under Hybrid Representations This repository contains authors' implementation of HybridPose: 6D Object Pose Esti

SONG, Chen 346 Jul 27, 2022
Simple inference deep head pose ncnn version

ncnn-deep-head-pose Simple implement inference deep head pose ncnn version with high performance and optimized resource. This project based on deep-he

Đỗ Công Minh 11 Jun 13, 2022
GFPGAN-ncnn - a naive NCNN implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration

GFPGAN-ncnn a naive ncnn implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration model support: 1.GFPGANClean

FeiGeChuanShu 32 Jul 13, 2022
A lightweight 2D Pose model can be deployed on Linux/Window/Android, supports CPU/GPU inference acceleration, and can be detected in real time on ordinary mobile phones.

A lightweight 2D Pose model can be deployed on Linux/Window/Android, supports CPU/GPU inference acceleration, and can be detected in real time on ordinary mobile phones.

JinquanPan 49 Aug 1, 2022
pose demo on android mobile based on PaddleDetection

pose_demo_android pose demo on android mobile based on PaddleDetection 本工程Android部分基于Paddle-Lite-Demo修改。 算法模型基于PaddleDetection的PP-TinyPose. 如欲获取更多详情,请

null 32 Aug 2, 2022
Android hair/human segmentation demo by ncnn

ncnn_Android_human Android hair/human segmentation demo by ncnn PS:performance maybe poor.it's just a demo:) Reference: 1.https://github.com/Tencent/n

null 23 May 23, 2022
PiDiNet running in Android by ncnn

PiDiNet-NCNN 使用NCNN在安卓上实现PiDiNet这个边缘检测网络。 什么是PiDiNet?指南针:https://github.com/zhuoinoulu/pidinet 文件构成 ncnn-android-nanodet是基于nihui的nanodet项目魔改的,被我改成了运行P

WuJinxuan 6 Jul 25, 2022
monodepth running in Android by ncnn

monodepth-NCNN 将wavelet-monodepth的模型搬运到NCNN上,工程里面给了安卓的工程以及以及生成好的app安装包 wavelet-monodepth wavelet-monodepth:RGB图像的深度估计,wavelet顾名思义,就使用了小波变换的,官方的工程在这:ht

WuJinxuan 14 Jul 11, 2022