MobileNet Image Classification with ESP32-CAM and Edge Impulse (TinyML)

Overview

MobileNet Image Classification on ESP32-CAM and Edge Impulse (TinyML)

41Ub2S0SjXL AC

This example is for running a MobileNet neural network model on a 10-dollar Ai-Thinker ESP32-CAM board, preferably with a ESP32-CAM-MB programmer board (or use an external USB-TTL module, see here). You also need to install Arduino-ESP32 board support in Arduino IDE and select Ai Thinker ESP32-CAM.

This is modified from ESP32 Cam and Edge Impulse with simplified code and copied necessary libraries from Espressif's esp-face. esp-face had been changed a lot into esp-dl and thus broke the original example. The original example also requires WiFi and the user has to refresh the webpage to classify a new image.

See the original example repo or this article about how to generate your own model on Edge Impulse. You can also still run the original example by copy every libraries in this example to the project directory, then re-open the .ino script.

You can also enable the flashlight by uncommenting the last two lines in setup().

This example does not output image via a web server, it can be modified to run in anyway you like, as long as you can point the camera to the images's direction. Be noted that you won't be able to read anything in the serial port monitor if you use Arduino IDE 2.0!

The Example Model - Cat & Dog Classification

I used Microsoft's Kaggle Cats and Dogs Dataset which has 12,500 cats and 12,500 dogs. 24,969 photos had successfully uploaded and split into 80-20% training/test sets.

The model I choose was MobileNetV1 96x96 0.25 (no final dense layer, 0.1 dropout) with transfer learning. Since free Edge Impulse accounts has a training time limit of 20 minutes per job, I can only train the model for 5 cycles. (You can go ask for more though...) I imagine if you have only a dozen images per class, you can try better models or longer training cycles.

Anyway, I got 89.8% accuracy for training set and 86.97% for test set, which seems to be decent enough.

1

Also, ESP32-CAM is not yet an officially supported board, so I cannot use EON Tuner for futher find-tuning.

You can find my published Edge Impulse project here: esp32-cam-cat-dog.

ei-esp32-cam-cat-dog-arduino-1.0.4.zip is the downloaded Arduino library which can be imported into Ardiono IDE.

The camera captures 240x240 images and resize them into 96x96. The inference time is 2607 ms (2.6 secs) per image, which is not very fast, with mostly good results. I don't know yet if different image sets or models may effect the result.

bogdan-farca-CEx86maLUSc-unsplash

richard-brutyo-Sg3XwuEpybU-unsplash

Owner
Alan Wang
So long and thanks for all the fi...I mean feedback. Email: [email protected]
Alan Wang
DeepI2P - Image-to-Point Cloud Registration via Deep Classification. CVPR 2021

#DeepI2P: Image-to-Point Cloud Registration via Deep Classification Summary Video PyTorch implementation for our CVPR 2021 paper DeepI2P. DeepI2P solv

Li Jiaxin 114 May 4, 2022
Spying on Microcontrollers using Current Sensing and embedded TinyML models

Welcome to CurrentSense-TinyML CurrentSense-TinyML is all about detecting microcontroller behaviour with current sensing and TinyML. Basically we are

Santander Security Research 66 Mar 11, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.5k May 9, 2022
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

Microsoft 13.8k May 17, 2022
Training and Evaluating Facial Classification Keras Models using the Tensorflow C API Implemented into a C++ Codebase.

CFace Training and Evaluating Facial Classification Keras Models using the Tensorflow C API Implemented into a C++ Codebase. Dependancies Tensorflow 2

null 8 Nov 23, 2021
Learning embeddings for classification, retrieval and ranking.

StarSpace StarSpace is a general-purpose neural model for efficient learning of entity embeddings for solving a wide variety of problems: Learning wor

Facebook Research 3.8k May 8, 2022
fast face classification

Fast Face Classification (F²C)—— An Efficient Training Approach for Very Large Scale Face Recognition Training on ultra-large-scale datasets is time-c

null 54 Apr 28, 2022
a linux cam solution to create gcode from cad-model

kutecam Note: work in progress! a linux cam solution to create gcode from cad-model Requirements Qt5 opencascade 7.6 cmake build system Wiki contains

Django Reinhard 4 Apr 14, 2022
LIDAR(Livox Horizon) point cloud preprocessing, including point cloud filtering and point cloud feature extraction (edge points and plane points)

LIDAR(Livox Horizon) point cloud preprocessing, including point cloud filtering and point cloud feature extraction (edge points and plane points)

hongyu wang 10 Apr 12, 2022
TensorFlow Lite, Coral Edge TPU samples (Python/C++, Raspberry Pi/Windows/Linux).

TensorFlow Lite, Coral Edge TPU samples (Python/C++, Raspberry Pi/Windows/Linux).

Nobuo Tsukamoto 84 May 8, 2022
Edge ML Library - High-performance Compute Library for On-device Machine Learning Inference

Edge ML Library (EMLL) offers optimized basic routines like general matrix multiplications (GEMM) and quantizations, to speed up machine learning (ML) inference on ARM-based devices. EMLL supports fp32, fp16 and int8 data types. EMLL accelerates on-device NMT, ASR and OCR engines of Youdao, Inc.

NetEase Youdao 173 May 6, 2022
GPTPU: General-Purpose Computing on (Edge) Tensor Processing Units

GPTPU: General-Purpose Computing on (Edge) Tensor Processing Units Welcome to the repository of ESCAL @ UCR's GPTPU project! We aim at demonstrating t

Extreme Storage and Computer Architecture Lab 31 Apr 30, 2022
EdgeKiller is a simple application that fully replaces Microsoft Edge with the Browser of choice.

EdgeKiller EdgeKiller is a simple application that fully replaces Microsoft Edge with the Browser of choice, while also intercepting all the microsoft

Jan Ochwat 2 Nov 30, 2021
Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, SRMD, RealSR, Anime4K, RIFE, CAIN, DAIN and ACNet.

Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, SRMD, RealSR, Anime4K, RIFE, CAIN, DAIN and ACNet.

Aaron Feng 6.7k May 17, 2022
Super Mario Remake using C++, SFML, and Image Processing which was a project for Structure Programming Course, 1st Year

Super Mario Remake We use : C++ in OOP concepts SFML for game animations and sound effects. Image processing (Tensorflow and openCV) to add additional

Omar Elshopky 4 Nov 20, 2021
NVIDIA Texture Tools samples for compression, image processing, and decompression.

NVTT 3 Samples This repository contains a number of samples showing how to use NVTT 3, a GPU-accelerated texture compression and image processing libr

NVIDIA DesignWorks Samples 21 Apr 24, 2022
Insight Toolkit (ITK) is an open-source, cross-platform toolkit for N-dimensional scientific image processing, segmentation, and registration

ITK: The Insight Toolkit C++ Python Linux macOS Windows Linux (Code coverage) Links Homepage Download Discussion Software Guide Help Examples Issue tr

Insight Software Consortium 1k May 16, 2022
An open source iOS framework for GPU-based image and video processing

GPUImage Brad Larson http://www.sunsetlakesoftware.com @bradlarson [email protected] Overview The GPUImage framework is a BSD-licensed iO

Brad Larson 19.9k May 3, 2022
ncnn demo of DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction

DocTr-ncnn ncnn demo of DocTr: Document Image Transformer for Geometric Unwarping and Illumination Correction model support: 1.Document Segmentation 2

FeiGeChuanShu 19 May 10, 2022