Automatic Number Plate Recognition (ANPR) system is a technology, which captures the number plate of a vehicle at a particular place cameras. The system can be implemented in places which mainly require security and can be implemented in the toll plaza, for parking fee collection, etc. [1-3]. The ANR carries a significant role in different applications, for example, traffic monitoring on the highway [4, 5], automatic toll fee collection [6], parking lots access control [7], identification of plundered vehicles [8], and crime detection systems [9, 10]. Though ANPR is a very old research area in image processing, it is still evolving year by year, because detecting the number plate from the image or from video is not an easy task as counting the vehicle from a stream of video [6, 9, 10]. Different researchers came with their own algorithm to detect the number plate, but each has some limitations; (1)- for some images, existing ANPR systems work perfectly, and for some images, it does not; (2)- next challenge is the diversity in the number plate writing style [11]; (3)- images captured under the severe weather and poor lighting conditions; (4)- low camera resolution; (5)- ANPR systems work better using the standard number plate, however, it becomes very tough to identify if the number plate has no standard size and pattern [2, 3].Therefore, this area is still growing and still imperfect. It requires a competent algorithm for better performance. Table 1 show different notations used in this paper.

The first ANPR system for vehicle identification was invented in 1976 at the Police Scientific Development Branch in the United Kingdom [12]. However, it has grown more and more interest during the last decade, along with the improvement of digital ANPR camera and the on going increase in computational capacity. This system was simply capable of automatically extracting and recognizing the vehicle number plate’s characters from an image.

A typical ANPR system has predefined four basic steps, as shown in Fig. (1), to recognize the number plate as explained in various research and journal papers [7, 8, 10].

1. Image Capture: In this step, vehicle image is captured by any standard camera or by extracting the interested frame from a stream of video retrieved from CCTV live footage.
2. Image Preprocessing: In this step, the captured image is further enhanced. Such enhancement includes resizing the image resolution, noise removal from the image, and conversion of the image from RGB to grayscale and then Binary (black and white).
3. Region of Interest (ROI): In this step, the specific number plate region is obtained from the enhanced image from Step 2. The extracted region is called ROI.
4. Optical character recognition (OCR): Electronic conversion of handwritten or printed text images into machine-encoded text. Here, OCR is used to recognize the number from the ROI image. A typical block diagram for ANR is shown in Fig. (2).

ANPR system has been suggested in different countries, including India [13, 14], Australia [15, 16], Malaysia [17, 18], America [19-21], Iran [22, 23], and Bangladesh [24, 25]. However, in the literature, very limited work exists for ANPR System for Pakistani vehicles [26-28]. Moreover, Pakistani number plate comes under the authority of each province and territories. There are eight types of number plates used in Pakistan. All number plates use the Latin alphabets in theory, but in practice, we see plates with Oriental script as well. Therefore, in this study, a new ANPR system has been implemented for vehicle number plate recognition targeting Pakistan vehicles. In this approach, after the preliminary steps, the last processing to be performed is OCR, where we adopted the template matching technique. Template matching was adapted as it is an efficient method for character recognition. Different challenges are addressed in the proposed ANPR system for Pakistani vehicles. First, the proposed ANPR system has to deal with a variety of license plates as the format and style of numbers varies from vehicle to vehicle and from region to region in Pakistan.

Fig. (2) shows the variety of vehicle license number plates as used across Pakistan. In the proposed ANPR system, we try our best to address these issues and to identify the number plate correctly.

The main contributions in this paper are summarized as follows: (1) - a new real-time framework to perform ANPR for Pakistani vehicles; (2) - a variety of vehicles from different regions of Pakistan were collected for experimenting the proposed ANPR System; (3) - the proposed ANPR was also experimented using different types of vehicles including Car, Jeep, Truck, Long Vehicles, Bus; (4) - finally, this study also presents a comparative analysis of the proposed approach with some existing ANPR approaches designed specifically for Pakistani license plates.

The remainder of the paper is organized as follows: different existing ANPR methodologies are discussed in Section 2, Related Work; Problem formulation, the proposed conceptual ANPR approach to the detection and recognition mechanism are discussed in Section 3. The performance evaluations of the proposed ANPR approach and the comparative analysis with the existing ANPR approaches are given in Section 4. This paper is closed with a conclusion and future work in Section 5, followed by the References.

In this section, we provide a detailed review of the different ANPR frameworks in general, while focusing on the work proposed in the perspective of Pakistan.

The system for automatic car license plate recognition includes a camera, a frame grabber, a computer, and custom-designed software for image processing, analysis, and recognition. In the literature, many methods have been proposed for the automatic VNPR system. Some of these studies have focused on NP region extraction only and others have proposed methods for NP region extraction as well as for recognition of text present in the NP. Some of these existing studies are briefly reviewed in the following paragraphs. In addition to this, interested readers are further advised to study the detailed survey on ANPR in previous studies [7, 8].

Fig. (1). A typical block diagram for an ANPR.

Fig. (2). Different license number plate designused in Pakistan.

A segmentation free automatic license plate recognition method, based on a Convolutional neural network, has been proposed in a study [29]. They used convolutional neural network layers to extract the features of the license number plate and then mapped this layer to fully connected layers with seven branches. Each branch out of the seven branches represent a specialized classifier for a character that resides on a specific position of the input license plate image. This work was very helpful because the possible errors generated in the segmentation step can be minimized and end-to-end support can also be provided.

Automatic number plate detection, for an intelligent transport system, by using an image processing technique has been proposed in a study [30]. The image processing techniques include counter matching and edge detection. An average accuracy of 96.67% has been achieved in the proposed method. The detection of license number plate, based on the color edge detection method, with different channel scale-space techniques, has been proposed in a study [31]. The aspect ratio is calculated by the proposed method and the licensed number plate is illustrious to detect the vehicle. Their work used the CIE-XYZ color model for process and accuracy of 94.23% was achieved.

A convolutional neural work based in the MD-YOLO framework for the multi-directional car license number plate detection has been proposed in a previous study [32]. This work uses fast intersection-over-union evaluation and precise rotation angle prediction strategy. The experimental results show that the proposed method achieved good results in terms of low computational cost and accuracy.

Mahamad et al. [33] suggested a simplified Malaysian vehicle plate number recognition. The techniques that have been used are digital image processing and OCR method. The proposed method does not give any information about the accuracy performance. A CNN-based MD-YOLO framework, for multi-directional car license plate detection has been proposed in a study [34]. The proposed method used precise rotation angle prediction and a fast intersection-over-union evaluation strategy. The experimental results show that the proposed method outclasses other current methods in terms of better accuracy and lower computational cost. Another work on ANPR was proposed by Mukherjee et al. [35], suggesting a robust algorithm for the extraction of image morphology, spatial filtering, and mapping features and characters that are used for vehicle number plate recognition. The proposed method has achieved an average accuracy of 79.30%.

This study is an effort to develop a low power and cost-efficient ANPR system for Pakistani vehicles, where the license number plates do not follow a particular pattern and people design and devise their own styles, which, of course, is not legally allowed, but is practiced. The proposed ANPR algorithm in this study discovers and recognized a vehicles’ license number plate automatically. The proposed algorithm is shown below and the block diagram of the proposed ANPR method is shown in Fig. (3).

Fig. (3). Block diagram of the proposed ANPR approach.

Fig. (4). An input image from CCTV recording to the ANPR system.

Fig. (5). Grayscale image retrieved from the original input image.

In the subsequent section, the proposed ANPR approach with the essential steps is discussed.

3.2. Conversion of Original Image to Grayscale

In this step, the original image is converted to grayscale because grayscale images are processed faster than the colored images. The color image is converted to 256-grayscale image using the following equation [22].

(1)

Where, A_GL= the converted gray level image A_R, A_G, A_B are the R, G, and B spectrum of the colored image, respectively. After gray scaling, blurring is applied in order to remove the noise and distortion in the image. Gaussian is applied on the gray scale image and useless details and noise are deleted from this image. Gaussian blur 2-d is shown in equation 2.

(2)

Where, i denotes the distance computed from the origin along the horizontal axis, j denotes the distance as calculated from the origin along the vertical axis, whereas Gaussian distribution standard deviation is ϑ. A convolution matrix is built by using the values of Gaussian distribution so it can be applied in the original image. The new value of each pixel is set to a weighted average of neighborhood pixels. The original pixels will be allocated to the highest weight and a smaller weight will be allocated to the neighboring pixel. A smoothed image will be produced, preserving boundaries and edges better than others. For example, the grayscale image from the original input image is given by Fig. (5) when Gaussian is applied.

Fig. (6). Edge detection process after applying the canny edge detection algorithm.

Fig. (7). Detection of the number region on the license number plate.

3.4. Candidate Number Plate Area Detection

After edge detection and some refining, we need to find all the contours in the image in order to take care of only those regions of the number plate where the contour width is more than the height. This is due to the fact that in the license number plate, normally, the width is more than its height. After the preprocessing step, the number plate region from the image is extracted. Now the marked area is the detected plate location as it is shown in Fig. (7).

Fig. (8). Cropping the marked area on the license number plate.

Fig. (9). Character segmentation.

3.7. Character Recognition System Using the KNN Based Approach

After the segmentation of elements (characters and numbers), the final module in the recognition process is character recognition. In the last module, character recognition is done by utilizing a template matching technique [38]. During this phase, the gray scale image is binarized by using binarization algorithm, after that, the blurring is applied to overcome the noise and distortion and finally, each character in the image is segmented into a separate image. Furthermore, each segmented character from the will be matched with the stored templates of the character. These stored templates consist of pixels. These pixels are termed as test point. There are two types of test points. One test point is called white test point and the other is termed as a black test point. The white area of the character image from the segmentation stage is tested by the white test point. The black test point is used to test the black area of the character image. The result of the test point is given by:

(3)

Matched Point = White test points fall in white pixel of the character image and black test points fall in the black pixel of the character image [37]. A predefined value has been allocated for black test points (80%) and white test points (90%). The character will be matched if the result is larger than these predefined values.

Finally, after the individual characters are separated, they are sent to the optical character recognition engine. Some common techniques of character recognition are pattern matching. The technique used by this system is KNN (supervised learning), which is a fairly simple algorithm and is capable of learning nonlinear high complex decision boundaries [39, 40]. In the next subsequent section, a detailed analysis of the proposed ANPR approach is presented to show the supremacy of the proposed system.

In order to validate the proposed method and exhibit its benefits, experiments on various Pakistan vehicle number plates have been carried out with comparisons. For testing, we have developed a database consisting of 900 images containing vehicles with very different number plate styles with different color and font style. The vehicle images are collected for different provinces of Pakistan. The captured images distribution against the different provinces/states is given in (Table 2).

Furthermore, the adopted parameters for the proposed ANPR approach during various stages are given below in Table 3.

We used different types of number plates out of which, 80% number plates had the standard license number plate visualization, while 20% of the plates did not follow the standard size, format and color scheme according to the region (Table 4).

The proposed ANPR approach achieved a maximum of 98% accuracy for the vehicles using the Islamabad Capital Territory license number plates. Whereas, the smallest accuracy was achieved in the case of Balochistan vehicle license number plates. Overall, the accuracy of the proposed ANPR approach remains 93%. In some cases, for instance, in the Baluchistan and Punjab, the accuracy was observed to be somewhat less than the others. This was due to different factors observed during the experimentations, including bright sunrays, dirty/unclear plates or blurring and non-standard vehicle number plates.

We also compared our approach with the existing ANPR approaches proposed for Pakistani vehicles. For this, we used different ANPR approaches suggested for Pakistani license number plates introduced in previous studies [26, 37]. For this comparison, we randomly selected 300, 400, and 900 license number plates from our total 900 number plates. The comparative analysis is shown in Table 5.

From the comparative analysis represented in Table 5, we can easily observe that the proposed ANPR system achieved more accuracy than the two existing ANPR approaches suggested specifically for the Pakistani license number plates. Moreover, the proposed ANPR for Pakistani license number plates showed more accuracy in a total of 400 vehicles. When we experimented on 900 vehicles, the accuracy was more than 5% as compared to the existing ANPR approaches.

Although there are a lot of running vehicle plate recognition systems for countries like China, Saudi Arabia, Korea, and Malaysia, there is no such LPR for Pakistani license plates. Although some work exists for Pakistani vehicle number plate identification it still needs more attention due to the variety of number plates and an increasing number of vehicles on the roads. The proposed research work is, therefore, targeting the identification and visualization of the Pakistani vehicle number plates. The proposed algorithm is tested over a huge image dataset, consisting of about 900 vehicles of different types and from various states of Pakistan. The collected images have captured in various illumination conditions and are verified to generate a high recognition rate. This research work is both a time and money-saving profit for law enforcement agencies and private organizations for improving homeland security. There is a need to expand the types of vehicles that can be detected: trucks, buses, scooters, bikes, etc. This technology can be further improved to detect the crashed vehicle’s number plate in an accident and alert the closest hospital and police station about the accident, thus saving lives.