This is a sample ncnn android project, it depends on ncnn library and opencv

Overview

The yolox object detection

This is a sample ncnn android project, it depends on ncnn library and opencv

https://github.com/Tencent/ncnn

https://github.com/nihui/opencv-mobile

how to build and run

step1

https://github.com/Tencent/ncnn/releases

  • Download ncnn-YYYYMMDD-android-vulkan.zip or build ncnn for android yourself
  • Extract ncnn-YYYYMMDD-android-vulkan.zip into app/src/main/jni and change the ncnn_DIR path to yours in app/src/main/jni/CMakeLists.txt

step2

https://github.com/nihui/opencv-mobile

  • Download opencv-mobile-XYZ-android.zip
  • Extract opencv-mobile-XYZ-android.zip into app/src/main/jni and change the OpenCV_DIR path to yours in app/src/main/jni/CMakeLists.txt

step3

  • Open this project with Android Studio, build it and enjoy!

some notes

  • Android ndk camera is used for best efficiency
  • Crash may happen on very old devices for lacking HAL3 camera interface
  • All models are manually modified to accept dynamic input shape
  • Most small models run slower on GPU than on CPU, this is common
  • FPS may be lower in dark environment because of longer camera exposure time

screenshot

reference

https://github.com/nihui/ncnn-android-nanodet
https://github.com/Megvii-BaseDetection/YOLOX

Issues
  • Question about ncnnoptimize model.param model.bin yolox.param yolox.bin 65536

    Question about ncnnoptimize model.param model.bin yolox.param yolox.bin 65536

    In: https://github.com/Megvii-BaseDetection/YOLOX/tree/main/demo/ncnn/cpp

    Step 5:

    suppose you are still under ncnn/build/tools/ncnn dir. ../ncnnoptimize model.param model.bin yolox.param yolox.bin 65536

    Suppose I trained a new customed model, how to adjust parameter (65536)? Thanks a lot.

    opened by liminghu 2
  • 在on_image_render函数下对图像rgb进行灰度化,结果会导致手机上闪退

    在on_image_render函数下对图像rgb进行灰度化,结果会导致手机上闪退

    具体代码

    void MyNdkCamera::on_image_render(cv::Mat& rgb) const
    {
        if (rgb.empty())
            return;
        cv::Mat src = rgb.clone();
        cv::Mat matGray;
        cv::cvtColor(src, matGray, CV_RGB2GRAY);
        {
            ncnn::MutexLockGuard g(lock);
            if (g_yolox)
            {
                std::vector<Object> objects;
                g_yolox->detect(rgb, objects);
                g_yolox->draw(rgb, objects);
            }
            else
            {
                draw_unsupported(rgb);
            }
        }
        draw_fps(rgb);
    }
    

    进行debug,具体报错

    `

    E/cv::error(): OpenCV(4.5.3) Error: Requested object was not found (could not open directory: /data/app/com.tencent.ncnnyolox-tvY0PkV6XrQyBx4J6SbZxw==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv-mobile/opencv-mobile/opencv-4.5.3/modules/core/src/glob.cpp, line 273
    A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 975 (ImageReader-640), pid 32479 (ncent.ncnnyolox)
    

    `

    opened by meixiansen 0
  • 在on_image_render函数下增加对输入图像rgb进行灰度化,结果会导致手机上闪退

    在on_image_render函数下增加对输入图像rgb进行灰度化,结果会导致手机上闪退

    具体代码 `void MyNdkCamera::on_image_render(cv::Mat& rgb) const { if (rgb.empty()) return; cv::Mat src = rgb.clone(); cv::Mat matGray; cv::cvtColor(src, matGray, CV_RGB2GRAY); // nanodet { ncnn::MutexLockGuard g(lock);

        if (g_yolox)
        {
            std::vector<Object> objects;
            g_yolox->detect(rgb, objects);
    
            g_yolox->draw(rgb, objects);
        }
        else
        {
            draw_unsupported(rgb);
        }
    }
    
    draw_fps(rgb);
    

    }进行debug,具体报错E/cv::error(): OpenCV(4.5.3) Error: Requested object was not found (could not open directory: /data/app/com.tencent.ncnnyolox-tvY0PkV6XrQyBx4J6SbZxw==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv-mobile/opencv-mobile/opencv-4.5.3/modules/core/src/glob.cpp, line 273 A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 975 (ImageReader-640), pid 32479 (ncent.ncnnyolox)`

    opened by meixiansen 0
  • YOLOX sluggish on GPU

    YOLOX sluggish on GPU

    Hey, I've a problem with GPU on newer devices like s20, poco x3 pro, etc. models: nano, tiny and s are faster on a CPU than on the GPU. On GPU models work slower (about 30-40%) and sometimes get sluggish (droping to 3 fps when on CPU I've got 15).

    opened by HyperScypion 0
  • ncnn-20210720-android-vulkan 编译出错

    ncnn-20210720-android-vulkan 编译出错

    下载ncnn-YYYYMMDD-android-vulkan.zip和下载opencv-mobile-XYZ-android.zip在ncnn-android-yolox项目中,编译后,opencv没问题,就是ncnn报错

    cmake的设置如此,几乎没有改动,出现这个错误,找不到ncnn的线程,文件的目录如下,要改改,麻烦告知,谢谢 1633694752 1633694766(1) 1633694784(1)

    opened by chenmingguo 1
  • Can't get custom network to perform in ncnn

    Can't get custom network to perform in ncnn

    Hi,

    1. I train a custom yolox-s network
    2. It works fine when testing with .\tools\demo.py
    3. I export the onnx model using simplify
    4. Convert to ncnn using onnx2ncnn
    5. I follow all instructions to replace the head of the .param file

    original: 7767517 235 268 Input images 0 1 images Split splitncnn_input0 1 4 images images_splitncnn_0 images_splitncnn_1 images_splitncnn_2 images_splitncnn_3 Crop Slice_4 1 1 images_splitncnn_3 467 -23309=1,0 -23310=1,2147483647 -23311=1,1 Crop Slice_9 1 1 467 472 -23309=1,0 -23310=1,2147483647 -23311=1,2 Crop Slice_14 1 1 images_splitncnn_2 477 -23309=1,0 -23310=1,2147483647 -23311=1,1 Crop Slice_19 1 1 477 482 -23309=1,1 -23310=1,2147483647 -23311=1,2 Crop Slice_24 1 1 images_splitncnn_1 487 -23309=1,1 -23310=1,2147483647 -23311=1,1 Crop Slice_29 1 1 487 492 -23309=1,0 -23310=1,2147483647 -23311=1,2 Crop Slice_34 1 1 images_splitncnn_0 497 -23309=1,1 -23310=1,2147483647 -23311=1,1 Crop Slice_39 1 1 497 502 -23309=1,1 -23310=1,2147483647 -23311=1,2 Concat Concat_40 4 1 472 492 482 502 503 0=0 Convolution Conv_41 1 1 503 877 0=32 1=3 11=3 2=1 12=1 3=1 13=1 4=1 14=1 15=1 16=1 5=1 6=3456

    Modified: 7767517 226 268 Input images 0 1 images Input focus 1 1 images 503 Convolution Conv_41 1 1 503 877 0=32 1=3 11=3 2=1 12=1 3=1 13=1 4=1 14=1 15=1 16=1 5=1 6=3456

    1. Then using ncnnoptimize with fp16 switch 65536
    2. Replace second dummy input with YoloV5Focus 7767517 220 250 Input images 0 1 images YoloV5Focus focus 1 1 images 503 Convolution Conv_41 1 1 503 877 0=32 1=3 4=1 5=1 6=3456 ...

    When doing this with yolox-tiny I end up with the exact same network as the provided demo one except the number of classes in the output which is 191 in my case.

    Now everything looks fine but the network does not perform at all - like not working at all. For training of the final network I use 640x640 images and this is used in training and also set in the app. I also tested tiny networks with 416x416 but always the same result.

    The environment I train and test is on Windows using onnx 1.10.1, onnxruntime 1.9.0, onnx-simplifier 0.3.6. Using yolox 0.1.0 and latest ncnn. I have tested almost everything over weeks now and just can't find what the issue could be. I also looked at other posts where they suggested to use onnx version 1.8.1, onnxruntime 1.8.0 and onnx-simplifier 0.3.5 - but also having this set in an environment it is the same result as well as doing the conversion on a linux box. Followed instructions here https://zhuanlan.zhihu.com/p/391788686 and others but always end up with the same non-performing network in ncnn.

    Please let me know what I'm missing. I'm aware that this is not an issue with this project.

    opened by realElMartino 4
RealSR-NCNN-Android is a simple Android application that based on Realsr-NCNN & Real-ESRGAN.

RealSR-NCNN-Android Real-ESRGAN is a Practical Algorithms for General Image Restoration. RealSR-NCNN-Android is a simple Android application that base

null 104 Jun 26, 2022
Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite, TensorRT, OpenCV, ncnn, MNN, SNPE, Arm NN, NNAbla

InferenceHelper This is a helper class for deep learning frameworks especially for inference This class provides an interface to use various deep lear

iwatake 146 Jun 19, 2022
GFPGAN-ncnn - a naive NCNN implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration

GFPGAN-ncnn a naive ncnn implementation of GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration model support: 1.GFPGANClean

FeiGeChuanShu 29 Jun 29, 2022
Mixed reality VR laser tag using Oculus Quest 2 and OAK-D depth cameras. First prize winner for North America region in OpenCV AI Competition 2021.

Mixed Reality Laser Tag Copyright 2021 Bart Trzynadlowski Overview This is the source code to my Mixed Reality Laser Tag project, which won first priz

null 34 Jun 3, 2022
Example of using ultralytics YOLO V5 with OpenCV 4.5.4, C++ and Python

yolov5-opencv-cpp-python Example of performing inference with ultralytics YOLO V5, OpenCV 4.5.4 DNN, C++ and Python Looking for YOLO V4 OpenCV C++/Pyt

null 108 Jun 29, 2022
Example of using YOLO v4 with OpenCV, C++ and Python

yolov4-opencv-cpp-python Example of performing inference with Darknet YOLO V4, OpenCV 4.4.0 DNN, C++ and Python Looking for YOLO V5 OpenCV C++/Python

null 38 May 19, 2022
cvnp: pybind11 casts between numpy and OpenCV, possibly with shared memory

cvnp: pybind11 casts and transformers between numpy and OpenCV, possibly with shared memory Explicit transformers between cv::Mat / cv::Matx and numpy

Pascal Thomet 6 Jun 8, 2022
GPU PyTorch TOP in TouchDesigner with CUDA-enabled OpenCV

PyTorchTOP This project demonstrates how to use OpenCV with CUDA modules and PyTorch/LibTorch in a TouchDesigner Custom Operator. Building this projec

David 65 Jun 15, 2022
A simple facial recognition script using OpenCV's FaceRecognizer module implemented in C++

Local Binary Patterns Histogram Recognizer A proyect that implements the LBPHRecognizer class of the OpenCV library to determine if a detected face co

Pablo Agustín Ortega-Kral 0 Jan 18, 2022
Libcamera with OpenCV in Raspberry Pi 64 bit Bullseye

Libcamera OpenCV RPi Bullseye 64OS Libcamera + OpenCV on a Raspberry Pi 4 with 64-bit Bullseye OS In the new Debian 11, Bullseye, you can only capture

Q-engineering 6 Apr 14, 2022
liteCV is greater than OpenCV :)

liteCV liteCV is lightweight image processing library for C++11. Unlike OpenCV, liteCV must be SIMPLE. Unlike OpenCV, liteCV must be INDEPENDENCE. Ach

Samuel Lee 3 May 26, 2022
Android hand detect and pose estimation by ncnn

ncnn_nanodet_hand 1.hand detect:用nanode-m训练了个hand detect模型, 2.hand pose:用CMU的数据集训练了个ghostnet作为backbone模仿pfld的handpose模型 3.推理:handpose.cpp单独检测pose,nano

null 68 May 11, 2022
Android face detect and segmentation by ncnn

Android face detect and segmentation by ncnn

null 95 Jun 24, 2022
Android hair/human segmentation demo by ncnn

ncnn_Android_human Android hair/human segmentation demo by ncnn PS:performance maybe poor.it's just a demo:) Reference: 1.https://github.com/Tencent/n

null 23 May 23, 2022
Android yolox hand detect by ncnn

The yolox hand detection This is a sample ncnn android project, it depends on ncnn library and opencv https://github.com/Tencent/ncnn https://github.c

FeiGeChuanShu 11 Apr 1, 2022
Android MoveNet pose estimation by ncnn

ncnn_Android_MoveNet Android MoveNet single human pose estimation by ncnn this project is a ncnn Android demo for MoveNet, it depends on ncnn library

FeiGeChuanShu 71 Jun 22, 2022
PiDiNet running in Android by ncnn

PiDiNet-NCNN 使用NCNN在安卓上实现PiDiNet这个边缘检测网络。 什么是PiDiNet?指南针:https://github.com/zhuoinoulu/pidinet 文件构成 ncnn-android-nanodet是基于nihui的nanodet项目魔改的,被我改成了运行P

WuJinxuan 5 Apr 29, 2022
monodepth running in Android by ncnn

monodepth-NCNN 将wavelet-monodepth的模型搬运到NCNN上,工程里面给了安卓的工程以及以及生成好的app安装包 wavelet-monodepth wavelet-monodepth:RGB图像的深度估计,wavelet顾名思义,就使用了小波变换的,官方的工程在这:ht

WuJinxuan 13 Jun 1, 2022
YOLOP running in Android by ncnn

YOLOP-NCNN 将YOLOP的模型搬运到NCNN上,工程里面给了windows下的VS测试以及安卓实现 YOLOP YOLOP:车辆检测+路面分割+车道线分割 三合一的网络,基于YOLO系列设计的,官方的工程在这:https://github.com/hustvl/YOLOP 工程细节 VS2

WuJinxuan 25 May 19, 2022