Builds and runs an exported image classification impulse on ESP32 Cam

Overview

ESP32 Cam and Edge Impulse

How to run custom inference on a ESP32 cam using Edge Impulse.

Material

esp32-cam

This code has been tested the AI Thinker ESP32 Cam module. It should work the same with the Wrover board or an board that has PSRAM.

To use this board, please select your board in the Arduino code the following lines:

// Select camera model

#define CAMERA_MODEL_WROVER_KIT // Has PSRAM
//#define CAMERA_MODEL_ESP_EYE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_PSRAM // Has PSRAM
//#define CAMERA_MODEL_M5STACK_V2_PSRAM // M5Camera version B Has PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_ESP32CAM // No PSRAM
//#define CAMERA_MODEL_AI_THINKER // Has PSRAM
//#define CAMERA_MODEL_TTGO_T_JOURNAL // No PSRAM

Steps

  • Create your Image Classification model using Edge Impulse.

Due to the board limitations, you may need to train your model with 48x48 images and use the MobileNetV2 0.05:

creat-impulse

  • Download the Arduino library under the Deployment tab in the Edge Impulse studio dl-arduino-lib

Basic Image Classification Example

Note: This project does not do a proper resize of the image capture but cutout the data

  • Open the Basic-Image-Classification.ino file under the /Basic-Image-Classification folder.
  • Import the .zip library you have downloaded from Edge Impulse Studio import-zip
  • Change the #include <esp32-cam_image-classification_inference.h> line according to your project name.
  • Compile and deploy the code to your board
  • Open the serial monitor and use the provided IP to capture an image and run the inference: serial-monitor inference

Advanced Image Classification Example

Note: Here we use the ESP SDK to resize the image in RGB888 format using the bilinear interpolation technique. You can see the funtion declaration on Espressif's Github repository.

  • Open the Basic-Image-Classification.ino file under the /Advanced-Image-Classification folder.
  • Set your WIFI credentials
  • Navigate to the app_httpd.cpp tab.
  • Import the .zip library you have downloaded from Edge Impulse Studio like on the previous example: import-zip
  • Change the #include <euros_coins_classification_inference.h> line according to your project name.
  • Compile and deploy the code to your board
  • Open the serial monitor and use the provided IP to capture an image and run the inference: serial-monitor
  • On your brower navigate to the IP provided by your Serial Console
  • Use the toggle button to activate the Edge Impulse Inference (you need to select an image resolution lower or equal to QVGA).
  • Click on Run inference: inference-50c inference-1e

Ressources

Note: Theses tutorials / repositories have been used to create this project:

Comments
  • Compilation error

    Compilation error

    Hello, I'm following the steps from Basic example, after compiling I get following error. Do you have any advice ?

    Basic-Image-Classification:4:10: fatal error: vector: No such file or directory #include ^~~~~~~~ compilation terminated. exit status 1 vector: No such file or directory

    bug 
    opened by Jakub-Bielawski 10
  • Unable to train Object Detection with 48x48 image size

    Unable to train Object Detection with 48x48 image size

    It seems that Edge Impulse may have changed something on their side, I am not able to train object detection for images with a size of 48x48, I will get an error. Any idea on when this will change? I really want this to work!

    opened by kevin192291 1
  • Esp32 CAM Serial Connection issue with edge impulse deamon cli

    Esp32 CAM Serial Connection issue with edge impulse deamon cli

    I am using esp32 cam module but facing issue in connecting serial with device and edge imple cli. It is giving an error of timeout. I want to understand is esp32 cam module is supported by edge impulse if not then how it is different from esp eye.

    opened by sunischit1 0
  • missing lib

    missing lib

    After following the guide for a simple object detector on the esp32 I hit

    vector: No such file or directory #include

    would be great if these examples were actually vetted and working...

    opened by jeremy-rutman 0
  • Missing libraries

    Missing libraries

    Looks like the esp-dl (https://github.com/espressif/esp-dl, former esp-face) repo has changed so much that many libraries went missing in this example. My temporary solution is go to this forked repo and copied all the .c and .h files under image_util and lib into the project dir to make it work.

    It would be great if you can update it to fit Espressif's new APIs.

    opened by alankrantas 0
  • Memory allocation error when deploying example model

    Memory allocation error when deploying example model

    Hello,

    We are trying to deploy the example model from the tutorial from this GitHub repository README (https://www.survivingwithandroid.com/tinyml-esp32-cam-edge-image-classification-with-edge-impulse/) on our ESP-EYE device.

    When using the most basic model (MobileNetV2 96x96 0.05) in Edge-Impulse the deployment works but the model is not accurate. Every other model fails with the following errors:

    1. When deploying the model with the default partitions scheme we are getting the following error: WiFi connected\n Starting web server on port: '80' Starting stream server on port: '81' Camera Ready! Use 'http://192.168.1.158' to connect Capture image Edge Impulse standalone inferencing (Arduino) ERR: Failed to run DSP process (-1002) run_classifier returned: -5

    2. When deploying the model in arduino IDE using the "Huge APP" partition scheme we are getting the following error: WiFi connected Starting web server on port: '80' Starting stream server on port: '81' Camera Ready! Use 'http://192.168.1.158' to connect Capture image Edge Impulse standalone inferencing (Arduino) ERR: failed to allocate tensor arena Failed to allocate TFLite arena (error code 1) run_classifier returned: -6

    The ESP-EYE has 4MB of memory available. According to the arduino IDE, the code itself takes ~1.2MB of memory. According to the Edge-Impulse website, all models do not need more than 1MB of additional memory. However, it seems that the memory is the issue here.

    Adding a screenshot of our board settings in arduino IDE: image

    Can you please advise on how can we make the more complicated models work on our device? Thank you!

    opened by karen-nativ 0
  • FRAMESIZE_240X240

    FRAMESIZE_240X240

    Advanced-Image-Classification:53:25: error: 'FRAMESIZE_240X240' was not declared in this scope
         config.frame_size = FRAMESIZE_240X240;
                             ^
    Advanced-Image-Classification:57:25: error: 'FRAMESIZE_240X240' was not declared in this scope
         config.frame_size = FRAMESIZE_240X240;
                             ^
    Advanced-Image-Classification:71:23: error: 'FRAMESIZE_240X240' was not declared in this scope
       s->set_framesize(s, FRAMESIZE_240X240);
    

    basic and advanced examples not compiling

    opened by kiralikbeyin 1
Owner
Edge Impulse
Enabling developers to create the next generation of intelligent device solutions through embedded Machine Learning
Edge Impulse
My humble attempt at getting tensorflow and the ESP32-CAM to cooperate

ESP32-Object-Recognition My humble attempt at getting tensorflow and the ESP32-CAM to cooperate (among other things) Notes: "Webcam_detection.py" isn'

null 13 Jun 21, 2022
Tiny and cheap robot car for inspecting sewer pipes >= 125 mm. With pan servo for the ESP32-Cam module

ESP32-Cam Sewer inspection car Version 1.0.0 - work in progress Based on esp32-cam-webserver by Owen Carter. Additional Features Pan servo for the ESP

Armin 5 Nov 6, 2022
esp32 cam digital low latency fpv

esp32-cam-fpv esp32 cam digital, low latency FPV This project uses a modified esp-camera component running on an AI Thinker board to send low-latency

null 126 Dec 31, 2022
ESP32-CAM with LVGL Speech/Face Recognition IR Control

ESP_MASTER 视频介绍:https://www.bilibili.com/video/BV1SM4y1V7Yb This is a comprehensive project that combines the powerful computing capabilities of ESP32

Kevincoooool 198 Jan 5, 2023
ESP32-Cam mod for Voron V2

ESP32-Cam Mod For Voron V2 This is a webcam mod for Voron V2. This project is currently under development. What else you need: ESP32-CAM with ov2640/o

Vaan Ip 33 Dec 10, 2022
Automatically exported from code.google.com/p/vartypes

======================================================================== VarTypes Author: Stefan Zickler <http://szickler.net>, (C) 2007-2015 Avai

null 14 Jan 22, 2022
My old heavily modified version of bigbase v1, it has an impulse-like scrollbar, ytd header loader, Vector3 fix + gamestate fix and some other misc changes!

Old Bigbase V1 UI This is my old ui for bigbase v1 but i dont need it anymore because the dev of solar mod menu stole it, and the new paragon menu (Fr

null 13 Sep 13, 2022
Aims to be an accurate C port of Impulse Tracker 2.15's IT replayer (with selectable IT2 sound drivers)

it2play Aims to be an accurate C port of Impulse Tracker 2.15's IT replayer (with selectable IT2 sound drivers). This is a direct port of the original

Olav Sørensen 18 Sep 27, 2022
ESP32-Skid-Steer - Bruder Catepillar Skid Steer model converted to RC, controlled by an ESP32 with 2 analog joysticks and a receiver that is an ESP32 on the model.

ESP32-Skid-Steer Bruder Catepillar Skid Steer model converted to RC, controlled by an ESP32 with 2 analog joysticks and a receiver that is an ESP32 on

null 6 Oct 27, 2022
ESP-32 CAM implementation of a Bird classifier using Arduino IDE

Bird-Classifier ESP-32 CAM implementation of a Bird classifier using Arduino IDE For the file server to run, you also need to have the ESP32 SD File M

null 3 Mar 28, 2022
null 313 Dec 31, 2022
Useful cmake macros that help with: compiler/linker flags, collecting sources, PCHs, Unity builds and other stuff.

ucm - useful cmake macros ucm is a collection of cmake macros that help with: managing compiler/linker flags collecting source files with grouping in

Viktor Kirilov 196 Dec 20, 2022
Custom kernel for sweet based on Delta, builds hosted on @sweet_epsilon on Telegram (GH releases are outdated)

Linux kernel ============ This file was moved to Documentation/admin-guide/README.rst Please notice that there are several guides for kernel develop

Udit Karode 14 Mar 29, 2022
My builds of suckless software

I recently switched to dwm from i3wm, it took me a while to get used to the dwm stuff and also the suckless way. After dwm, came st and then slstatus

Kushagra Jain 2 Nov 19, 2021
Builds atlas texture from a bunch of input images.

Atlasc @septag atlasc is a command-line program that builds atlas texture from a bunch of input images. Main Features Cross-platform. Runs on linux/ma

Sepehr Taghdisian 74 Aug 30, 2022
This FreeRTOS example builds a simple Timer application for Linux using the ESP-IDF

Supported Targets Linux This FreeRTOS example builds a simple Timer application for Linux using the ESP-IDF. Build Source the IDF environment as usual

null 13 Apr 4, 2022
Automated builds/mirrors of various PS3SDKs for Linux systems.

Working PS3SDK Binaries NOTICE: This repo is now deprecated. SDK builds have moved here, and SDK mirrors have moved here. Prepares and releases workin

null 3 Jan 29, 2022
Pre-compiled builds for CVE-2021-4034

CVE-2021-4034 Precompiled builds for CVE-2021-4034. Of course you shouldn't trust precompiled builds :) This release works slightly different: first a

null 13 Dec 27, 2022
Allows for multiple SwitchBot buttons and curtains to be controlled via MQTT sent to ESP32. ESP32 will send BLE commands to switchbots and return MQTT responses to the broker. Also supports Meter/Temp Sensor

SwitchBot-MQTT-BLE-ESP32 Switchbot local control using ESP32. no switchbot hub used/required. works with any smarthub that supports MQTT https://githu

null 343 Dec 27, 2022