Is your ALPR system reading at 99%?
Not too long ago I had a conversation with a law enforcement official at a show. After explaining some of the merits of our system, he asked "Can your system read at 99%". It struck me at that time how misleading these read statistics can be.
If we remove all the statisticians from the room, and have a conversation with a reasonable person, telling him we read at 99% would probably lead him to believe the following:
- If he drove through a parking lot with 100 cars, he would read 99 of the 100 cars correctly.
This is a reasonable interpretation of the statistic. Unfortunately, the reality is not quite as simple, and these figures are misleading at best.
Let's take a look at the numbers and see what they mean.
Accuracy rate
Here is where we pull out the big numbers, this is where the LPR industry is talking in the "90's", and the big dogs are barking 99%! Wow, that's impressive. What this number usually boils down to is: of all the plates read, how many did the system capture correctly.
Here is an example to put this into context: If a system captures 100 plates, and reads 99 of them accurately, you have a 99% read rate. Although this number sounds impressive, it does not tell the whole story. Here is an exaggeration to make a point: if I drive my LPR system through a parking lot with 100 cars, but the system manages to capture and read only one plate, I can turn around and claim a 100% read rate (which is better than 99%). That's because the plates the system did capture were read perfectly. The fact that it captures only one out of a hundred plates is not factored into the figure.
A second system driving through the same parking lot and capturing 80 plates, reading 60 of them properly, would only be able to claim a 75% read rate. That second system however did a much better job (60 plates read compared to 1), but has an accuracy rate of 75% (60/80). This is where the confusion comes from: 100% sounds much better than 75%, but in this case, the 75% would serve you much better.***What's missing here is the concept of "Capture Rate". This is the number very few manufacturers talk about, but is very important to understand.
Capture rate
The capture rate represents the number of plates the system detected and captured compared to all the plates it came across. It may have read some plates incorrectly, but they are captured nonetheless.
This number on its own can be equally misleading. Here is another exaggerated example to explain. We run through the same parking lot above and the system captures 99 of the 100 plates. However, it reads only half of them correctly. The system's capture rate at 99% is impressive, but its accuracy is at only 50%. A large amount of data is collected, but half of this data is inaccurate.
The capture rate is not as frequently quoted as the read rate. Part of the reason is that this number is difficult to determine, and even harder to stand by, as many factors can influence it from one application to the next. Capture rate is greatly affected by environmental conditions-things like dirty plates, snow, trailer hitches, bent plates, traffic conditions, lighting, variety of plates, and the list goes on. Claiming a capture rate is difficult at best, and if you do get a number, it may very well not represent what you should expect in your own application.
To summarize, accuracy rates require capture rates in order to be put into context and capture rates often vary by specific applications. So when you are sold on a 99% figure for LPR, please take it with a grain of salt.
On a side note, your best way to measure the performance of an LPR solution is to see a demonstration. Set the conditions for the demonstration as close to your real-world application as possible. If your climate has plenty of rain or snow, require that the demonstration be conducted in those conditions. If you plan to read plates at night, do the demo in the evening. If your LPR vendor has a solid product, he will be happy to arrange a demonstration for you, no matter what the conditions.