The edges of image are considered to be the most important image attributes that provide valuable information for human image perception. Edge detection is a type of image segmentation technique which is used to simplify the image data to minimize the amount of data to be processed which is required in the analysis of identification of an image. Edges are points in an image where there is a sharp transition in the pixel intensity level. In this project, Canny edge detection, one of the efficient edge detection algorithms is implemented on a Zedboard FPGA using verilog. The input image is stored on a PC and fed to the FPGA. The processed output image is displayed on a VGA monitor. We can harness the parallel processing capabilities of FPGA to implement a highly pipelined image processing architecture which suits for realtime applications.
The System Topology:
Canny Edge Detection:
The process of canny edge detection algorithm can be broken down to five different steps:
- Apply Gaussian filter to smooth the image in order to remove the noise
- Find the intensity gradients of the image using sobel operation.
- Apply gradient magnitude thresholding or lower bound cut-off suppression to get rid of spurious response to edge detection
- Apply double threshold to determine potential edges
- Track edge by hysteresis: Finalize the detection of edges by suppressing all the other edges that are weak and not connected to strong edges.
The image after a 3x3 Gaussian mask has been passed across each pixel. Since all edge detection results are easily affected by the noise in the image, it is essential to filter out the noise to prevent false detection caused by it. To smooth the image, a Gaussian filter kernel is convolved with the image. This step will slightly smooth the image to reduce the effects of obvious noise on the edge detector. It is important to understand that the selection of the size of the Gaussian kernel will affect the performance of the detector. The larger the size is, the lower the detector's sensitivity to noise. Additionally, the localization error to detect the edge will slightly increase with the increase of the Gaussian filter kernel size.
An edge in an image may point in a variety of directions, so the canny algorithm uses four filters to detect horizontal, vertical and diagonal edges in the blurred image. The edge detection operator (here sobel) returns a value for the first derivative in the horizontal direction (Gx) and the vertical direction (Gy). From this the edge gradient and direction can be determined: The edge direction angle is rounded to one of four angles representing vertical, horizontal and the two diagonals (0°, 45°, 90° and 135°). An edge direction falling in each color region will be set to a specific angle values, for instance θ in [0°, 22.5°] or [157.5°, 180°] maps to 0°.
3.Non Maximum Suppression
This is an edge thinning technique. Lower bound cut-off suppression is applied to find the locations with the sharpest change of intensity value. The algorithm for each pixel in the gradient image is:
a) Compare the edge strength of the current pixel with the edge strength of the pixel in the positive and negative gradient directions.
b) If the edge strength of the current pixel is the largest compared to the other pixels in the mask with the same direction (e.g., a pixel that is pointing in the y-direction will be compared to the pixel above and below it in the vertical axis), the value will be preserved. Otherwise, the value will be suppressed.
The algorithm categorizes the continuous gradient directions into a small set of discrete directions, and then moves a 3x3 filter over the output of the previous step (that is, the edge strength and gradient directions). At every pixel, it suppresses the edge strength of the center pixel (by setting its value to 0) if its magnitude is not greater than the magnitude of the two neighbors in the gradient direction. Note that the sign of the direction is irrelevant, i.e. north–south is the same as south–north and so on.
After application of non-maximum suppression, remaining edge pixels provide a more accurate representation of real edges in an image. However, some edge pixels remain that are caused by noise and color variation. In order to account for these spurious responses, it is essential to filter out edge pixels with a weak gradient value and preserve edge pixels with a high gradient value.
This is accomplished by selecting high and low threshold values. If an edge pixel’s gradient value is higher than the high threshold value, it is marked as a strong edge pixel. If an edge pixel’s gradient value is smaller than the high threshold value and larger than the low threshold value, it is marked as a weak edge pixel. If an edge pixel's gradient value is smaller than the low threshold value, it will be suppressed. The two threshold values are empirically determined and their definition will depend on the content of a given input image.
5.Edge Tracking by Hysteresis
So far, the strong edge pixels should certainly be involved in the final edge image, as they are extracted from the true edges in the image. However, there will be some debate on the weak edge pixels, as these pixels can either be extracted from the true edge, or the noise/color variations. To achieve an accurate result, the weak edges caused by the latter reasons should be removed.
Usually a weak edge pixel caused from true edges will be connected to a strong edge pixel while noise responses are unconnected. To track the edge connection, blob analysis is applied by looking at a weak edge pixel and its 8-connected neighborhood pixels. As long as there is one strong edge pixel that is involved in the blob, that weak edge point can be identified as one that should be preserved.
The Complete Block Design:
The canny edge detection framework has been designed using verilog and the top module is packaged as an IP. Then the block design is created for acquiring image, processing it and displaying it in a VGA monitor. Lot of IPs from Xilinx has been used. The input image is transferred from the PC to the external DDR of the Zedboard FPGA. The output processed image is also stored back to the DDR. AXI DMA IP from Xilinx is used to access the contents of the DDR memory. Zedboard belongs to Zync 7000 series which features a Zync processing system. The Zync processing system has Programmable Logic (PL) and the Processing Subsystem(PS) (ARM Cortex-A9). The PL is designed, synthesized and implemented in Vivado software. The PS is controlled using Vitis Unified Platform by writing C code.
Edge detection is pervasive in several applications such as finger print matching , medical diagnosis and license plate detection. Self driving cars rely on realtime edge detection to safely maneuver through the roads. This project can be extended to cater the needs of lot of these demanding applications by using the parallel processing potential of the FPGA.
https://slack-files.com/T023E787WLV-F02DV7BGXCJ-ddc1b40935 FPGA Implementation of Canny edge detection
https://automaticaddison.com/how-the-sobel-operator-works/ Basics of Image Processing
https://www.youtube.com/channel/UCXwcHOQ2ktHdERXNKCpmWnQ Basics of Interfacing
https://www.xilinx.com/support/documentation/ip_documentation/div_gen/ v5_1/pg151-div-gen.pdf Divider IP Manual
https://www.xilinx.com/support/documentation/user_guides/ug585-Zynq- 7000-TRM.pdf Zynq SoC Manual
https://usermanual.wiki/Pdf/ZedBoard20Users20Guide.1528655017/view ZedBoard User Manual
https://digilent.com/reference/programmable-logic/zedboard/reference- manual ZedBoard reference Manual
https://projectf.io/posts/fixed-point-numbers-in-verilog/ Fixed-point numbers in Verilog
https://www.fpgadeveloper.com/2014/08/using-the-axi-dma-in-vivado.html/ AXI DMA
https://towardsdatascience.com/canny-edge-detection-step-by-step-in- python-computer-vision-b49c3a2d8123 Understanding Canny Edge detection