A project demonstration on how to use the GigE camera to do the DeepStream Yolo3 object detection

Overview

  • SPDX-License-Identifier: NVIDIA Corporation
  • Copyright (c) 2019-2022. All rights reserved.
  • NVIDIA Corporation and its licensors retain all intellectual property
  • and proprietary rights in and to this software, related documentation
  • and any modifications thereto. Any use, reproduction, disclosure or
  • distribution of this software and related documentation without an express
  • license agreement from NVIDIA Corporation is strictly prohibited.

Instroduction

This deepstream-yolo3-gige application uses the Nvidia DeepStream-5.1 SDK to generate video or display on screen after object detection.

This apps uses the Basler acA1920-25gc GigE camera (ethernet port) with the ARAVIS SDK [1] as the plugin. Also it takes advantages of Nvidi DeepStream-5.1 SDK with the Yolo3 Libraries for quick DS-5.1 application.

With the GigE camera in place, this application idea can be extended into many applications, such as the highway trafic follow monitoring, industrial production line for quality control, supermarket safety control, etc.

What is ARAVIS?

Aravis is a glib/gobject based library for video acquisition using Genicam cameras. It currently implements the gigabit ethernet anbd USB3 protocols used by the industrial cameras. It also provides a basic ethernet camera simulator and simple video viewer[1]. License requirement is under GNU General Public License, Version 2

How to Install ARAVIS?

download the aravis-0.6.0 (this version works for this apps) and put it in /src dir. as example. One can try the latest version as well.

cd aravis-0.6.0
./configure --enable-viewer --prefix=/usr
make
make install
After get libgstaravis-0.6.so, one need to copy that file into /usr/lib/x86_64-linux-gnu for dGPU
for Jetson devices, for example of Jetson NX, copy that file into /usr/lib/aarch64-linux-gnu

Prequisites:

Please follow instructions in the apps/sample_apps/deepstream-app/README on how to install the prequisites for Deepstream SDK, the DeepStream SDK itself and the apps.

One must have the following development packages installed

DeepStream Pipeline

  • DeepStream SDK is based on the GStreamer framework. GStreamer is a pipeline based multimedia framework that links together a wide variety of media processing systems to complete workflows. Following is the pipleline for this Yolo3 GiGE application which is based on the DL Yolo3 network for the Object Dectection. gst-pipleline-png

This DeepStream Yolo3 GigE Apps Overview

  • This application takes advantages of Nvidia DeepStream-5.1 SDK on the Yolo3 Object Detection Libraries - no training needed

  • Nvidia Transfer Learning Toolkit 3.0 can be used for the re-train if one needs to use different dataset

  • The ARAVIS SDK and Basler GigE camera will be used for this application

  • TLT converter to generate the TRT engine file for the DeepStream Deployment for the Yolo3 GigE application if need re-train

  • User can make selection: either display the bounding box of object detection on screen or save as a encoded H264 video file

How to Compile the Application Package

  • git clone this application into /opt/nvidia/deeepstream/deepstream-5.1/sources/apps/sample_apps

  • $ cd deepstream-yolov3-gige

  • $ make

  • For Jetson edge device, e.g, Xavier NX, one need to use deepstream-yolov3-gige-jetson-nx.c

How to Run this DeepStream Yolo3 GiGE Application

  • $ example: sudo ./deepstream_yolov3_gige -- object detection results will be displayed on screen only

  • $ example: sudo ./deepstream_yolov3_gige out.h264 -- produce the video output file which is in encoded H264 format.

  • Example picture from the object detection result

yolo3-result

Nvidia Transfer Learning Toolkit 3.0 for Re-Training, Evaluation, Export, and Quick Deployment

  • TLT Converter Information (include how to download) : https://developer.nvidia.com/tlt-get-started

  • Use Nvidia TLT 3.0 for Re-Training, Evaluation, Export, and Quick Deployment

  • Use the TLT to generate the .etlt and .engine file for the DeepStream application deployment after retraining

  • One can refer to Jupyter Notebook (yolo_v3.ipynb) for Yolo3 training based on the new dataset used

  • Refer to default_spec as the configuration file used by the Jupyter Notebook for the training, evaluation, and export

  • Use the Jupyter Notebook to download the resenet_18 for yolov3 from Nvidia GPU Cloud : https://ngc.nvidia.com/

The information for Nvidia Transfer Learning Toolkit 3.0 User Guide for Yolo3

Quickly Deploying the Apps to DeepStream-5.1 Using Transfer Learning Toolkit-3.0

  • Use the .etlt or .engine file after TLT train, export, and coverter

  • Use the Jetson version of the tlt converter to generate the .engine file used in the Jetson devices

    example: ./tlt-converter -k $key -e trt.fp16.engine -t fp16 -p input_1 1,1x3x600x600, 4x3x600x600,16x3x600x600 frcnn_kitti_resnet50_retrain_fp16.etlt
    here: $key is the key when do the tlt train and 600x600 is the input/training image size as example

  • Define the .etlt or .engine file path in the config file for dGPU and Jetson for the DS-5.1 application

  • example: model-engine-file = trt.int8.engine in config_infer_primary_yoloV3.txt

  • The performance using different GPU devices

performance-jetson-dgpu

References

You might also like...
A simple demonstration of how PyTorch autograd works

简单地演示了 PyTorch 中自动求导机制的原理。 官方博客:https://pytorch.org/blog/overview-of-pytorch-autograd-engine/ 编译运行 使用 Bazel bazel run autograd_test 包含了一个使用 MSE 损失函数的一

ROS2 packages based on NVIDIA libArgus library for hardware-accelerated CSI camera support.
ROS2 packages based on NVIDIA libArgus library for hardware-accelerated CSI camera support.

Isaac ROS Argus Camera This repository provides monocular and stereo nodes that enable ROS developers to use cameras connected to Jetson platforms ove

The code implemented in ROS projects a point cloud obtained by a Velodyne VLP16 3D-Lidar sensor on an image from an RGB camera.
The code implemented in ROS projects a point cloud obtained by a Velodyne VLP16 3D-Lidar sensor on an image from an RGB camera.

PointCloud on Image The code implemented in ROS projects a point cloud obtained by a Velodyne VLP16 3D-Lidar sensor on an image from an RGB camera. Th

 LiDAR-Camera Calibration using 3D-3D Point correspondences
LiDAR-Camera Calibration using 3D-3D Point correspondences

ROS package to find a rigid-body transformation between a LiDAR and a camera for "LiDAR-Camera Calibration using 3D-3D Point correspondences"

YOLOV4 tiny + lane detection on Android with 8 FPS!
YOLOV4 tiny + lane detection on Android with 8 FPS!

YOLOV4 Tiny + Ultra fast lane detection on Android with 8 FPS! Tested with HONOR 20PRO Kirin 980

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation
OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

Build Type Linux MacOS Windows Build Status OpenPose has represented the first real-time multi-person system to jointly detect human body, hand, facia

A coverage-guided and memory-detection enabled fuzzer for windows applications.
A coverage-guided and memory-detection enabled fuzzer for windows applications.

WDFuzzer Manual 中文手册见 README_CN.md WDFuzzer:winafl + drmemory WDFuzzer is an A coverage-guided and memory detection abled fuzzer for for windows softw

A modern object detector inside fragment shaders
A modern object detector inside fragment shaders

YOLOv4 Tiny in UnityCG/HLSL Video Demo: https://twitter.com/SCRNinVR/status/1380238589238206465?s=20 Overview YOLOv4 Tiny is one of the fastest object

The SCRFD face detection, depends on ncnn library and opencv
The SCRFD face detection, depends on ncnn library and opencv

The SCRFD face detection, depends on ncnn library and opencv

Comments
  • Could not create 'src_elem'

    Could not create 'src_elem'

    When I try to run the deepstrem_yolo3_gige I get this:

    Could not create 'src_elem' Failed to create source bin. Exiting.

    I've been trying to solve it but nothing seems to work. Any idea of why could this happen?

    Thanks in advance.

    opened by unaaimendi 1
Owner
NVIDIA AI IOT
NVIDIA AI IOT
deploy yolox algorithm use deepstream

YOLOX(Megvii-BaseDetection) Deploy DeepStream ?? ?? This project base on https://github.com/Megvii-BaseDetection/YOLOX and https://zhuanlan.zhihu.com/

null 78 Nov 18, 2022
3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint

3D-Lidar Camera Calibration using planar Point to to camera Plane Constraint

Subodh Mishra 74 Nov 11, 2022
YOLOv4 accelerated wtih TensorRT and multi-stream input using Deepstream

Deepstream 5.1 YOLOv4 App This Deepstream application showcases YOLOv4 running at high FPS throughput! P.S - Click the gif to watch the entire video!

Akash James 35 Nov 10, 2022
Training and fine-tuning YOLOv4 Tiny on custom object detection dataset for Taiwanese traffic

Object Detection on Taiwanese Traffic using YOLOv4 Tiny Exploration of YOLOv4 Tiny on custom Taiwanese traffic dataset Trained and tested AlexeyAB's D

Andrew Chen 4 Oct 3, 2022
Official PyTorch Code of GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection (CVPR 2021)

GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Mo

Abhinav Kumar 76 Nov 6, 2022
UAV images dataset for moving object detection

PESMOD PESMOD (PExels Small Moving Object Detection) dataset consists of high resolution aerial images in which moving objects are labelled manually.

İbrahim Delibaşoğlu 34 Oct 26, 2022
Object Based Generic Perception Object Model

This model is a highly parameterizable generic perception sensor and tracking model. It can be parameterized as a Lidar or a Radar. The model is based on object lists and all modeling is performed on object level.

TU Darmstadt - FZD 5 Jun 11, 2022
Real-time object detection with YOLOv5 and TensorRT

YOLOv5-TensorRT The goal of this library is to provide an accessible and robust method for performing efficient, real-time inference with YOLOv5 using

Noah van der Meer 37 Nov 19, 2022
Port of the 2020 support library to Raspberry Pi for the VL53L3CX Time-of-Flight ranging sensor with advanced multi-object detection

Port of ST VL53L3CX (2020) driver library to Raspberry Pi This is a port of the support library to Raspberry Pi for the VL53L3CX Time-of-Flight rangin

Niall Douglas 4 Jul 27, 2022
YOLOX + ROS2 object detection package

YOLOX-ROS YOLOX+ROS2 Foxy Supported List Base ROS1 C++ ROS1 Python ROS2 C++ ROS2 Python CPU ✅ CUDA ✅ CUDA (FP16) ✅ TensorRT (CUDA) ✅ OpenVINO ✅ MegEng

Ar-Ray 144 Nov 8, 2022