Canny edge detection, one of the efficient edge detection algorithms is implemented on a Zedboard FPGA using verilog

Overview

canny_edge_detection_in_FPGA

Abstract:

The edges of image are considered to be the most important image attributes that provide valuable information for human image perception. Edge detection is a type of image segmentation technique which is used to simplify the image data to minimize the amount of data to be processed which is required in the analysis of identification of an image. Edges are points in an image where there is a sharp transition in the pixel intensity level. In this project, Canny edge detection, one of the efficient edge detection algorithms is implemented on a Zedboard FPGA using verilog. The input image is stored on a PC and fed to the FPGA. The processed output image is displayed on a VGA monitor. We can harness the parallel processing capabilities of FPGA to implement a highly pipelined image processing architecture which suits for realtime applications.

The System Topology:

Canny_BlockDiagram

Canny Edge Detection:

The process of canny edge detection algorithm can be broken down to five different steps:

  1. Apply Gaussian filter to smooth the image in order to remove the noise
  2. Find the intensity gradients of the image using sobel operation.
  3. Apply gradient magnitude thresholding or lower bound cut-off suppression to get rid of spurious response to edge detection
  4. Apply double threshold to determine potential edges
  5. Track edge by hysteresis: Finalize the detection of edges by suppressing all the other edges that are weak and not connected to strong edges.

1.Gaussian Blur

The image after a 3x3 Gaussian mask has been passed across each pixel. Since all edge detection results are easily affected by the noise in the image, it is essential to filter out the noise to prevent false detection caused by it. To smooth the image, a Gaussian filter kernel is convolved with the image. This step will slightly smooth the image to reduce the effects of obvious noise on the edge detector. It is important to understand that the selection of the size of the Gaussian kernel will affect the performance of the detector. The larger the size is, the lower the detector's sensitivity to noise. Additionally, the localization error to detect the edge will slightly increase with the increase of the Gaussian filter kernel size.

Gaussian_kernel

2.Sobel Operation

An edge in an image may point in a variety of directions, so the canny algorithm uses four filters to detect horizontal, vertical and diagonal edges in the blurred image. The edge detection operator (here sobel) returns a value for the first derivative in the horizontal direction (Gx) and the vertical direction (Gy). From this the edge gradient and direction can be determined: The edge direction angle is rounded to one of four angles representing vertical, horizontal and the two diagonals (0°, 45°, 90° and 135°). An edge direction falling in each color region will be set to a specific angle values, for instance θ in [0°, 22.5°] or [157.5°, 180°] maps to 0°.

Sobel_kernel

3.Non Maximum Suppression

This is an edge thinning technique. Lower bound cut-off suppression is applied to find the locations with the sharpest change of intensity value. The algorithm for each pixel in the gradient image is:

a) Compare the edge strength of the current pixel with the edge strength of the pixel in the positive and negative gradient directions.

b) If the edge strength of the current pixel is the largest compared to the other pixels in the mask with the same direction (e.g., a pixel that is pointing in the y-direction will be compared to the pixel above and below it in the vertical axis), the value will be preserved. Otherwise, the value will be suppressed.

The algorithm categorizes the continuous gradient directions into a small set of discrete directions, and then moves a 3x3 filter over the output of the previous step (that is, the edge strength and gradient directions). At every pixel, it suppresses the edge strength of the center pixel (by setting its value to 0) if its magnitude is not greater than the magnitude of the two neighbors in the gradient direction. Note that the sign of the direction is irrelevant, i.e. north–south is the same as south–north and so on.

4.Double Thresholding

After application of non-maximum suppression, remaining edge pixels provide a more accurate representation of real edges in an image. However, some edge pixels remain that are caused by noise and color variation. In order to account for these spurious responses, it is essential to filter out edge pixels with a weak gradient value and preserve edge pixels with a high gradient value.

This is accomplished by selecting high and low threshold values. If an edge pixel’s gradient value is higher than the high threshold value, it is marked as a strong edge pixel. If an edge pixel’s gradient value is smaller than the high threshold value and larger than the low threshold value, it is marked as a weak edge pixel. If an edge pixel's gradient value is smaller than the low threshold value, it will be suppressed. The two threshold values are empirically determined and their definition will depend on the content of a given input image.

5.Edge Tracking by Hysteresis

So far, the strong edge pixels should certainly be involved in the final edge image, as they are extracted from the true edges in the image. However, there will be some debate on the weak edge pixels, as these pixels can either be extracted from the true edge, or the noise/color variations. To achieve an accurate result, the weak edges caused by the latter reasons should be removed.

Usually a weak edge pixel caused from true edges will be connected to a strong edge pixel while noise responses are unconnected. To track the edge connection, blob analysis is applied by looking at a weak edge pixel and its 8-connected neighborhood pixels. As long as there is one strong edge pixel that is involved in the blob, that weak edge point can be identified as one that should be preserved.

The Complete Block Design:

The canny edge detection framework has been designed using verilog and the top module is packaged as an IP. Then the block design is created for acquiring image, processing it and displaying it in a VGA monitor. Lot of IPs from Xilinx has been used. The input image is transferred from the PC to the external DDR of the Zedboard FPGA. The output processed image is also stored back to the DDR. AXI DMA IP from Xilinx is used to access the contents of the DDR memory. Zedboard belongs to Zync 7000 series which features a Zync processing system. The Zync processing system has Programmable Logic (PL) and the Processing Subsystem(PS) (ARM Cortex-A9). The PL is designed, synthesized and implemented in Vivado software. The PS is controlled using Vitis Unified Platform by writing C code.

Block_design

Results Obtained:

op_1

op_2

op_3

Conclusion

Edge detection is pervasive in several applications such as finger print matching , medical diagnosis and license plate detection. Self driving cars rely on realtime edge detection to safely maneuver through the roads. This project can be extended to cater the needs of lot of these demanding applications by using the parallel processing potential of the FPGA.

References

  1. https://slack-files.com/T023E787WLV-F02DV7BGXCJ-ddc1b40935 FPGA Implementation of Canny edge detection

  2. https://automaticaddison.com/how-the-sobel-operator-works/ Basics of Image Processing

  3. https://www.youtube.com/channel/UCXwcHOQ2ktHdERXNKCpmWnQ Basics of Interfacing

  4. https://www.xilinx.com/support/documentation/ip_documentation/div_gen/ v5_1/pg151-div-gen.pdf Divider IP Manual

  5. https://www.xilinx.com/support/documentation/user_guides/ug585-Zynq- 7000-TRM.pdf Zynq SoC Manual

  6. https://usermanual.wiki/Pdf/ZedBoard20Users20Guide.1528655017/view ZedBoard User Manual

  7. https://digilent.com/reference/programmable-logic/zedboard/reference- manual ZedBoard reference Manual

  8. https://projectf.io/posts/fixed-point-numbers-in-verilog/ Fixed-point numbers in Verilog

  9. https://www.fpgadeveloper.com/2014/08/using-the-axi-dma-in-vivado.html/ AXI DMA

  10. https://towardsdatascience.com/canny-edge-detection-step-by-step-in- python-computer-vision-b49c3a2d8123 Understanding Canny Edge detection

You might also like...
ROS package for visualizing Open Motion Planning Library (OMPL) algorithms in 2D with Rviz.
ROS package for visualizing Open Motion Planning Library (OMPL) algorithms in 2D with Rviz.

OMPL 2D Rviz Visualizer Visualizing, animating and debugging Open Motion Planning Library (OMPL) algorithms in ROS Rviz. The package allows OMPL plann

CXXGraph is a small library, header only, that manages the Graph and it's algorithms in C++.
CXXGraph is a small library, header only, that manages the Graph and it's algorithms in C++.

CXXGraph is a small library, header only, that manages the Graph and it's algorithms in C++.

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities
Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Sketch-Based Streaming Anomaly Detection in Dynamic Graphs
Sketch-Based Streaming Anomaly Detection in Dynamic Graphs

AnoGraph Implementation of Sketch-Based Streaming Anomaly Detection in Dynamic Graphs. Siddharth Bhatia, Mohit Wadhwa, Philip S. Yu, Bryan Hooi Existi

 FFVideo - an example FFmpeg lib, and wxWidgets Player with video filters and face detection
FFVideo - an example FFmpeg lib, and wxWidgets Player with video filters and face detection

An example FFmpeg lib, and wxWidgets Player application with video filters and face detection, it is a no-audio video player intended for video experiments and developers learning how to code media applications.

Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc.
Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc.

Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc.

A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.
A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

A multi core friendly rigid body physics and collision detection library suitable for games and VR applications.

RGL - 3D visualization device system for R using OpenGL
RGL - 3D visualization device system for R using OpenGL

RGL - 3D visualization device system for R using OpenGL INTRODUCTION The RGL package is a visualization device system for R, using OpenGL or WebGL as

SVG animation from multiple SVGs or single GIF using tracer
SVG animation from multiple SVGs or single GIF using tracer

svgasm svgasm is a proof-of-concept SVG assembler to generate a self-contained animated SVG file from multiple still SVG files with CSS keyframes anim

Owner
Jeffrey Samuel
Jeffrey Samuel
Teracube 2e (4.19.y) - Extremely bleeding edge, you have been warned

Linux kernel ============ There are several guides for kernel developers and users. These guides can be rendered in a number of formats, like HTML an

Arcadia Open Source 4 Dec 16, 2022
A basic 3D scene implemented with various engines, frameworks or APIs.

Here be dragons Hic sunt dracones. This repository contains multiple implementations of the same 3D scene, using different APIs and frameworks on vari

Simon Rodriguez 1.7k Dec 27, 2022
My computer graphics playground. Currently has a raytracer implemented with D3D11 compute shader.

Graphics Playground I use this project as my "toy" engine. I'll be implementing various graphics projects in this repository. The code here is not sui

Berk Emre Sarıbaş 4 Aug 26, 2021
The official Open-Asset-Importer-Library Repository. Loads 40+ 3D-file-formats into one unified and clean data structure.

Open Asset Import Library (assimp) A library to import and export various 3d-model-formats including scene-post-processing to generate missing render

Open Asset Import Library 8.6k Jan 4, 2023
My implementations of Ray Tracing in One Weekend written in many different languages.

Ray Tracing in Many Languages This repository contains my implementation of the Ray Tracing in One Weekend book written in several different programmi

Joshua Vega 1 Oct 19, 2021
HARFANG®3D is an all-in-one 3D visualization library usable in C++, Python, Lua and Go.

HARFANG® 3D engine HARFANG®3D is an all-in-one 3D visualization library usable in C++, Python, Lua and Go. Table of contents About Features Screenshot

HARFANG® 3D 280 Jan 1, 2023
2D lowpoly editor heavily based on this one

polyedit 2D lowpoly editor heavily based on this one Download Direct link Releases page Libraries This project uses: SFML Tiny File Dialogs jsoncpp im

null 19 Aug 7, 2021
Simple and efficient screen recording utility for Windows.

simple and efficient screen recording utility for Windows

Mārtiņš Možeiko 484 Dec 31, 2022
Tiny and efficient graph abstractions.

Tiny and efficient graph abstractions. Usage See tinygraph-example.c Interface See documentation in tinygraph.h Building The tinygraph library require

tinygraph 18 Dec 14, 2022
SMAA is a very efficient GPU-based MLAA implementation (DX9, DX10, DX11 and OpenGL)

SMAA is a very efficient GPU-based MLAA implementation (DX9, DX10, DX11 and OpenGL), capable of handling subpixel features seamlessly, and featuring an improved and advanced pattern detection & handling mechanism.

Jorge Jimenez 848 Dec 30, 2022