OCR Recognition Errors

[tyliec]

Context

Currently, we collect our data from the PDFs using OCR. This leads to many OCR errors, as our image cropping isn't the strictest (+ there is no validation). This can lead to the problem that we are serving false data and statistics. An easy way to see this occurring is in our "Age" chart, where there are some sub 18 year old arrests (which obviously is not correct). This isn't a big issue, as the OCR works a good majority of the time (plus, we provide the raw data and record images for anyone to validate our statistics). However, it is still an issue that should be dealt with as we want to be as accurate as possible.

Potential Solutions

1. Have some sort of "validation" flag, with validated records vs. unvalidated records. This way we aren't presenting false statistics, as we can sort through validated/unvalidated data
2. Have a stricter bound around the fields we are trying to parse out of each record (Line by Line, instead of cropping over the entire section)

[tyliec]

This is not fully corrected. Closing though since we are now reading above 95% accuracy.