Automatic processing of scanned documents

Aim

Speed up the processing and verification of documents, as well as reduce the cost of the process, while maintaining high accuracy of data recognition.

Context:

Marketing Logic received an order from a company whose work is tightly tied to the processing of various types of documentation, including hand-filled questionnaires. One of the tasks of the client's employees is to compare data from questionnaires, scans of passports and contracts, check these documents for discrepancies and errors, and transform the data into an electronic format. Prior to the introduction of the Marketing Logic product, this process was carried out manually and, depending on the type of work, took from 10 to 100 FTE, and the verification of one set of documents took up to three days. Due to the significant amount of work, the management had to maintain a large staff, while due to the high workload and the human factor, employees made mistakes.

Solution

Marketing Logic has studied the documentation that the client works with — in this case, these are scans of passports of different quality, as well as forms of questionnaires and contracts, filled in, including handwritten text. Some of the digitized materials, along with their transcripts, were uploaded to the system Action.Docs, so that it can establish a correlation between the text displayed in the photo and the manually typed text. Based on the collected data, our team set up a text recognition model —so that the program most accurately interprets a certain fragment of an image as a number, letter, or symbol. In addition, we have developed an interface for the client's employees who process the documentation. With this software, the scanned materials can be uploaded to the system with a couple of clicks — it will process the data itself and provide a transcript.

Among other things, the Action.Docs system is able to detect its own errors — this option is especially useful when deciphering handwritten text. If the system does not know how to recognize a single fragment of a document, whether it is a word or a symbol, it informs the employee about it — and he enters the data manually. The program remembers the new ratio of the image and the transcript, adds it to the dictionary, and then interprets the same fragment on its own.

The same applies to cases where the system is not sure that it has correctly deciphered the text. Before starting work, employees set the minimum percentage of matching the image and the text stored in the dictionary, at which the fragment is considered processed. If the system considers that the accuracy of recognition of an individual fragment is below the specified threshold, the employee is asked to enter the correct value by himself. In addition, the Action.Docs system is able to perform lexical and semantic analysis of the decrypted text, and if the resulting interpretation seems illogical from the point of view of meaning, it also reports this to the employee. Thus, the system continues to self-learn throughout the entire time of its use.

Another Action.Docs feature is final classification of materials. The system can combine individual pages into a single pdf-file or, on the contrary, divide a multi-page document into separate files. If necessary, it also rotates the documents independently, puts them in the right folders and gives the files the correct names.

Result

Thanks to the Action.Docs system the cost of processing and checking client documentation scans has been reduced by 30-95%. All stages of processing a single set of documents — passport scans, application forms, and contracts-using the Marketing Logic product were reduced to a few seconds. At the same time, the accuracy of recognition of printed text, barcodes and hand marks was 99%, and handwritten text-up to 76%.