Eyes on the Target: Super-Resolution and License-Plate Recognition in Low-Quality Surveillance Videos

Low-quality surveillance cameras throughout the cities could provide important cues to identify a suspect, for example, in a crime scene. However, the license-plate recognition is especially difficult under poor image resolutions. In this vein, super-resolution (SR) can be an inexpensive solution, via software, to overcome this limitation. Consecutive frames in a video may contain different information that could be integrated into a single image, richer in details. In this paper, we design and develop a novel, free and open-source framework underpinned by SR and automatic license-plate recognition (ALPR) techniques to identify license-plate characters in low-quality real-world traffic videos, captured by cameras not designed specifically for the ALPR task, aiding forensic analysts in understanding an event of interest. The framework handles the necessary conditions to identify a target license plate, using a novel methodology to locate, track, align, super-resolve, and recognize its alphanumerics. The user receives as outputs the rectified and super-resolved license-plate, richer in detail, and also the sequence of license-plates characters that have been automatically recognized in the super-resolved image. Additionally, we also design and develop a novel SR method that projects the license-plates separately onto the rectified grid, and then fill in the missing pixels using inpainting techniques. We compare the different algorithms in the framework (five for tracking, three for registration, seven for reconstruction, two for post-processing, and two for the recognition step), and present discussions on the pros and cons of each choice. Our experiments show that SR can indeed increase the number of correctly recognized characters posing the framework as an important step toward providing forensic experts and practitioners with a solution for the license-plate recognition problem under difficult acquisition conditions.