An Efficient Approach for Vehicle Number Plate Recognition in Pakistan
Saif Ur Rehman1, *, Moiz Ahmad1, Asif Nawaz1, Tariq Ali1
Identifiers and Pagination:Year: 2020
First Page: 12
Last Page: 21
Publisher Id: TOAIJ-6-12
Article History:Received Date: 21/03/2020
Revision Received Date: 17/04/2020
Acceptance Date: 02/05/2020
Electronic publication date: 18/06/2020
Collection year: 2020
open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: (https://creativecommons.org/licenses/by/4.0/legalcode). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Recognition of Vehicle License Number Plates (VLNP) is an important task. It is valuable in numerous applications, such as entrance admission, security, parking control, road traffic control, and speed control. An ANPR (Automatic Number Plate Recognition) is a system in which the image of the vehicle is captured through high definition cameras. The image is then used to detect vehicles of any type (car, van, bus, truck, and bike, etc.), its’ color (white, black, blue, etc.), and its’ model (Toyota Corolla, Honda Civic etc.). Furthermore, this image is processed using segmentation and OCR techniques to get the vehicle registration number in form of characters. Once the required information is extracted from VLNP, this information is sent to the control center for further processing.
ANPR is a challenging problem, especially when the number plates have varying sizes, the number of lines, fonts, background diversity, etc. Different ANPR systems have been suggested for different countries, including Iran, Malaysia, and France. However, only a limited work exists for Pakistan vehicles. Therefore, in this study, we aim to propose a novel ANPR framework for Pakistan VLNP recognition.
The proposed ANPR system functions in three different steps: (i) - Number Plate Localization (NPL); (ii)- Character Segmentation (CS); and (iii)- Optical Character Recognition (OCR), involving template-matching mechanism. The proposed ANPR approach scans the number plate and instantly checks against database records of vehicles of interest. It can further extract the real=time information of driver and vehicle, for instance, license of the driver and token taxes of vehicles are paid or not, etc.
Finally, the proposed ANPR system has been evaluated on several real-time images from various formats of number plates practiced in Pakistan territory. In addition to this, the proposed ANPR system has been compared with the existing ANPR systems proposed specifically for Pakistani licensed number plates.
The proposed ANPR Model has both time and money-saving profit for law enforcement agencies and private organizations for improving homeland security. There is a need to expand the types of vehicles that can be detected: trucks, buses, scooters, bikes. This technology can be further improved to detect the crashed vehicle’s number plate in an accident and alert the closest hospital and police station about the accident, thus saving lives.
Automatic Number Plate Recognition (ANPR) system is a technology, which captures the number plate of a vehicle at a particular place cameras. The system can be implemented in places which mainly require security and can be implemented in the toll plaza, for parking fee collection, etc. [1-3]. The ANR carries a significant role in different applications, for example, traffic monitoring on the highway [4, 5], automatic toll fee collection , parking lots access control , identification of plundered vehicles , and crime detection systems [9, 10]. Though ANPR is a very old research area in image processing, it is still evolving year by year, because detecting the number plate from the image or from video is not an easy task as counting the vehicle from a stream of video [6, 9, 10]. Different researchers came with their own algorithm to detect the number plate, but each has some limitations; (1)- for some images, existing ANPR systems work perfectly, and for some images, it does not; (2)- next challenge is the diversity in the number plate writing style ; (3)- images captured under the severe weather and poor lighting conditions; (4)- low camera resolution; (5)- ANPR systems work better using the standard number plate, however, it becomes very tough to identify if the number plate has no standard size and pattern [2, 3].Therefore, this area is still growing and still imperfect. It requires a competent algorithm for better performance. Table 1 show different notations used in this paper.
|ANPR||Automated Number Plate Recognition|
|OCR||Optical Character Reader|
|RGB||Red Green Blue|
|ROI||Region Of Interest|
The first ANPR system for vehicle identification was invented in 1976 at the Police Scientific Development Branch in the United Kingdom . However, it has grown more and more interest during the last decade, along with the improvement of digital ANPR camera and the on going increase in computational capacity. This system was simply capable of automatically extracting and recognizing the vehicle number plate’s characters from an image.
- Image Capture: In this step, vehicle image is captured by any standard camera or by extracting the interested frame from a stream of video retrieved from CCTV live footage.
- Image Preprocessing: In this step, the captured image is further enhanced. Such enhancement includes resizing the image resolution, noise removal from the image, and conversion of the image from RGB to grayscale and then Binary (black and white).
- Region of Interest (ROI): In this step, the specific number plate region is obtained from the enhanced image from Step 2. The extracted region is called ROI.
- Optical character recognition (OCR): Electronic conversion of handwritten or printed text images into machine-encoded text. Here, OCR is used to recognize the number from the ROI image. A typical block diagram for ANR is shown in Fig. (2).
ANPR system has been suggested in different countries, including India [13, 14], Australia [15, 16], Malaysia [17, 18], America [19-21], Iran [22, 23], and Bangladesh [24, 25]. However, in the literature, very limited work exists for ANPR System for Pakistani vehicles [26-28]. Moreover, Pakistani number plate comes under the authority of each province and territories. There are eight types of number plates used in Pakistan. All number plates use the Latin alphabets in theory, but in practice, we see plates with Oriental script as well. Therefore, in this study, a new ANPR system has been implemented for vehicle number plate recognition targeting Pakistan vehicles. In this approach, after the preliminary steps, the last processing to be performed is OCR, where we adopted the template matching technique. Template matching was adapted as it is an efficient method for character recognition. Different challenges are addressed in the proposed ANPR system for Pakistani vehicles. First, the proposed ANPR system has to deal with a variety of license plates as the format and style of numbers varies from vehicle to vehicle and from region to region in Pakistan.
Fig. (2) shows the variety of vehicle license number plates as used across Pakistan. In the proposed ANPR system, we try our best to address these issues and to identify the number plate correctly.
The main contributions in this paper are summarized as follows: (1) - a new real-time framework to perform ANPR for Pakistani vehicles; (2) - a variety of vehicles from different regions of Pakistan were collected for experimenting the proposed ANPR System; (3) - the proposed ANPR was also experimented using different types of vehicles including Car, Jeep, Truck, Long Vehicles, Bus; (4) - finally, this study also presents a comparative analysis of the proposed approach with some existing ANPR approaches designed specifically for Pakistani license plates.
The remainder of the paper is organized as follows: different existing ANPR methodologies are discussed in Section 2, Related Work; Problem formulation, the proposed conceptual ANPR approach to the detection and recognition mechanism are discussed in Section 3. The performance evaluations of the proposed ANPR approach and the comparative analysis with the existing ANPR approaches are given in Section 4. This paper is closed with a conclusion and future work in Section 5, followed by the References.
2. BACKGROUND AND RELATED WORK
In this section, we provide a detailed review of the different ANPR frameworks in general, while focusing on the work proposed in the perspective of Pakistan.
The system for automatic car license plate recognition includes a camera, a frame grabber, a computer, and custom-designed software for image processing, analysis, and recognition. In the literature, many methods have been proposed for the automatic VNPR system. Some of these studies have focused on NP region extraction only and others have proposed methods for NP region extraction as well as for recognition of text present in the NP. Some of these existing studies are briefly reviewed in the following paragraphs. In addition to this, interested readers are further advised to study the detailed survey on ANPR in previous studies [7, 8].
|Fig. (1). A typical block diagram for an ANPR.|
|Fig. (2). Different license number plate designused in Pakistan.|
A segmentation free automatic license plate recognition method, based on a Convolutional neural network, has been proposed in a study . They used convolutional neural network layers to extract the features of the license number plate and then mapped this layer to fully connected layers with seven branches. Each branch out of the seven branches represent a specialized classifier for a character that resides on a specific position of the input license plate image. This work was very helpful because the possible errors generated in the segmentation step can be minimized and end-to-end support can also be provided.
Automatic number plate detection, for an intelligent transport system, by using an image processing technique has been proposed in a study . The image processing techniques include counter matching and edge detection. An average accuracy of 96.67% has been achieved in the proposed method. The detection of license number plate, based on the color edge detection method, with different channel scale-space techniques, has been proposed in a study . The aspect ratio is calculated by the proposed method and the licensed number plate is illustrious to detect the vehicle. Their work used the CIE-XYZ color model for process and accuracy of 94.23% was achieved.
A convolutional neural work based in the MD-YOLO framework for the multi-directional car license number plate detection has been proposed in a previous study . This work uses fast intersection-over-union evaluation and precise rotation angle prediction strategy. The experimental results show that the proposed method achieved good results in terms of low computational cost and accuracy.
Mahamad et al.  suggested a simplified Malaysian vehicle plate number recognition. The techniques that have been used are digital image processing and OCR method. The proposed method does not give any information about the accuracy performance. A CNN-based MD-YOLO framework, for multi-directional car license plate detection has been proposed in a study . The proposed method used precise rotation angle prediction and a fast intersection-over-union evaluation strategy. The experimental results show that the proposed method outclasses other current methods in terms of better accuracy and lower computational cost. Another work on ANPR was proposed by Mukherjee et al. , suggesting a robust algorithm for the extraction of image morphology, spatial filtering, and mapping features and characters that are used for vehicle number plate recognition. The proposed method has achieved an average accuracy of 79.30%.
3. METHODOLOGY - A PROPOSED APPROACH FOR AUTOMATIC NUMBER PLATE RECOGNITION
This study is an effort to develop a low power and cost-efficient ANPR system for Pakistani vehicles, where the license number plates do not follow a particular pattern and people design and devise their own styles, which, of course, is not legally allowed, but is practiced. The proposed ANPR algorithm in this study discovers and recognized a vehicles’ license number plate automatically. The proposed algorithm is shown below and the block diagram of the proposed ANPR method is shown in Fig. (3).
|Fig. (3). Block diagram of the proposed ANPR approach.|
|Fig. (4). An input image from CCTV recording to the ANPR system.|
|Fig. (5). Grayscale image retrieved from the original input image.|
In the subsequent section, the proposed ANPR approach with the essential steps is discussed.
3.1. Loading the Vehicle Image
The image is captured and passed to the system. After the plate location in the vehicle image is identified, it is sent to the next phase for conversion to the required scale. An example of an input image from CCTV recording for the ANPR system is shown in Fig. (4).
3.2. Conversion of Original Image to Grayscale
In this step, the original image is converted to grayscale because grayscale images are processed faster than the colored images. The color image is converted to 256-grayscale image using the following equation .
Where, AGL= the converted gray level image AR, AG, AB are the R, G, and B spectrum of the colored image, respectively. After gray scaling, blurring is applied in order to remove the noise and distortion in the image. Gaussian is applied on the gray scale image and useless details and noise are deleted from this image. Gaussian blur 2-d is shown in equation 2.
Where, i denotes the distance computed from the origin along the horizontal axis, j denotes the distance as calculated from the origin along the vertical axis, whereas Gaussian distribution standard deviation is
|Fig. (6). Edge detection process after applying the canny edge detection algorithm.|
|Fig. (7). Detection of the number region on the license number plate.|
3.3. Edge Detection
Next, the proposed ANPR system detects the edges in the grayscale image to be extracted. There are different methods, including Sobel Edge Detection, Roberts Edge Detection, Canny Edge Detection, Prewitt edge detection. In this study, canny operator was used to find the image edge. The Canny operator performs a 2-D spatial gradient measurement on the image. It applies two thresholds to the gradient: a high threshold for low edge sensitivity and a low threshold for high edge sensitivity. Edge starts with the low sensitivity result and then grows it to include connected edge pixels from the high sensitivity result. This helps fill in gaps in the detected edges. This edge detection process, using the canny operator, is represented in the Fig. (6).
3.4. Candidate Number Plate Area Detection
After edge detection and some refining, we need to find all the contours in the image in order to take care of only those regions of the number plate where the contour width is more than the height. This is due to the fact that in the license number plate, normally, the width is more than its height. After the preprocessing step, the number plate region from the image is extracted. Now the marked area is the detected plate location as it is shown in Fig. (7).
|Fig. (8). Cropping the marked area on the license number plate.|
|Fig. (9). Character segmentation.|
3.5. Cropping Number Plate Area
The next step is to crop the marked area as a new image and apply some processing to the newly obtained image. This will help to easily visualize and read each and every character in the new image as it is depicted in Fig. (8).
3.6. Character Segmentation
Character segmentation is the procedure of extracting the characters and numbers from the license plate image. Diverse aspects make the character segmentation task complicated, like image noise, plate frame, space mark, plate’s rotation, and light variance. A number of procedures have been proposed for character segmentation to overcome these aspects. We used the procedure proposed in a previous study . After each marked character is segmented into separate images, these are then forwarded to the KNN based character recognition system shown in Fig. (9).
3.7. Character Recognition System Using the KNN Based Approach
After the segmentation of elements (characters and numbers), the final module in the recognition process is character recognition. In the last module, character recognition is done by utilizing a template matching technique . During this phase, the gray scale image is binarized by using binarization algorithm, after that, the blurring is applied to overcome the noise and distortion and finally, each character in the image is segmented into a separate image. Furthermore, each segmented character from the will be matched with the stored templates of the character. These stored templates consist of pixels. These pixels are termed as test point. There are two types of test points. One test point is called white test point and the other is termed as a black test point. The white area of the character image from the segmentation stage is tested by the white test point. The black test point is used to test the black area of the character image. The result of the test point is given by:
Matched Point = White test points fall in white pixel of the character image and black test points fall in the black pixel of the character image . A predefined value has been allocated for black test points (80%) and white test points (90%). The character will be matched if the result is larger than these predefined values.
Finally, after the individual characters are separated, they are sent to the optical character recognition engine. Some common techniques of character recognition are pattern matching. The technique used by this system is KNN (supervised learning), which is a fairly simple algorithm and is capable of learning nonlinear high complex decision boundaries [39, 40]. In the next subsequent section, a detailed analysis of the proposed ANPR approach is presented to show the supremacy of the proposed system.
In order to validate the proposed method and exhibit its benefits, experiments on various Pakistan vehicle number plates have been carried out with comparisons. For testing, we have developed a database consisting of 900 images containing vehicles with very different number plate styles with different color and font style. The vehicle images are collected for different provinces of Pakistan. The captured images distribution against the different provinces/states is given in (Table 2).
|Province Region||Number of License Number Plates|
|Islamabad Capital Territory||150|
Furthermore, the adopted parameters for the proposed ANPR approach during various stages are given below in Table 3.
|Image Capturing Time||Morning|
|Distance||5 to 10 Meter|
|Camera Used||5 Mega Pixel|
|Type of Number plates||Mixed (Standard/non-standard)|
We used different types of number plates out of which, 80% number plates had the standard license number plate visualization, while 20% of the plates did not follow the standard size, format and color scheme according to the region (Table 4).
|State Name/Total Vehicles||Total Number of Plates Correctly Extracted||Accuracy (%)|
|Islamabad Capital Territory||147||98|
The proposed ANPR approach achieved a maximum of 98% accuracy for the vehicles using the Islamabad Capital Territory license number plates. Whereas, the smallest accuracy was achieved in the case of Balochistan vehicle license number plates. Overall, the accuracy of the proposed ANPR approach remains 93%. In some cases, for instance, in the Baluchistan and Punjab, the accuracy was observed to be somewhat less than the others. This was due to different factors observed during the experimentations, including bright sunrays, dirty/unclear plates or blurring and non-standard vehicle number plates.
We also compared our approach with the existing ANPR approaches proposed for Pakistani vehicles. For this, we used different ANPR approaches suggested for Pakistani license number plates introduced in previous studies [26, 37]. For this comparison, we randomly selected 300, 400, and 900 license number plates from our total 900 number plates. The comparative analysis is shown in Table 5.
|ANPR Approach||Accuracy (%)|
|Number of License Plates Used|
|Tahir et al. ,||90||91||88|
|Haider et al. ,||91||92||86|
|The Proposed ANPR||92||94||93|
From the comparative analysis represented in Table 5, we can easily observe that the proposed ANPR system achieved more accuracy than the two existing ANPR approaches suggested specifically for the Pakistani license number plates. Moreover, the proposed ANPR for Pakistani license number plates showed more accuracy in a total of 400 vehicles. When we experimented on 900 vehicles, the accuracy was more than 5% as compared to the existing ANPR approaches.
Although there are a lot of running vehicle plate recognition systems for countries like China, Saudi Arabia, Korea, and Malaysia, there is no such LPR for Pakistani license plates. Although some work exists for Pakistani vehicle number plate identification it still needs more attention due to the variety of number plates and an increasing number of vehicles on the roads. The proposed research work is, therefore, targeting the identification and visualization of the Pakistani vehicle number plates. The proposed algorithm is tested over a huge image dataset, consisting of about 900 vehicles of different types and from various states of Pakistan. The collected images have captured in various illumination conditions and are verified to generate a high recognition rate. This research work is both a time and money-saving profit for law enforcement agencies and private organizations for improving homeland security. There is a need to expand the types of vehicles that can be detected: trucks, buses, scooters, bikes, etc. This technology can be further improved to detect the crashed vehicle’s number plate in an accident and alert the closest hospital and police station about the accident, thus saving lives.
CONSENT FOR PUBLICATION
AVAILABILITY OF DATA AND MATERIALS
The source code is available on the following WebURL https://github.com/saifi009/An-Efficient-Approach-for-Vehicle-Number-Plate-Recognition-in-Pakistan/blob/master/Pakistan VehicleLicenceNumberPlate%20RecgnitionProjectSourceCode.rar
CONFLICT OF INTEREST
The authors declare no conflict of interest, financial or otherwise.