LLM-Aided OCR
lastmod 2024-08-16created 2024-08-161 edits
Enhance Tesseract OCR output for scanned PDFs by applying Large Language Model (LLM) corrections.
Correcting Tesseract OCR errors with LLMs
- Designed to significantly improve the quality of optical character recognition (OCR) output
- Uses modern natural language processing techniques and large language models (LLMs) to turn raw OCR text into highly accurate, well-formatted, readable documents
Features
- Converts PDFs to images
- OCR using Tesseract
- Advanced error correction using LLMs (local or API-based)
- Smart text chunking for efficient processing
- Markdown formatting option
- Header and page number suppression (optional)
- Quality assessment of the final output
- Supports local LLMs and cloud-based API providers (OpenAI, Anthropic)
- Asynchronous processing for better performance
- Detailed logging for process tracking and debugging
- GPU acceleration for local LLM inference
Requirements
- Python 3.12+
- Tesseract OCR engine
- PDF2Image library
- PyTesseract
- OpenAI API (optional)
- Anthropic API (optional)
- Local LLM support (optional, requires a compatible GGUF model)
How it works
The LLM-Aided OCR project uses a multi-stage process to turn raw OCR output into high-quality, readable text:
- PDF conversion: converts the input PDF to images using pdf2image
- OCR: applies Tesseract OCR to extract text from the images
- Text chunking: splits the raw OCR output into manageable chunks
- Error correction: each chunk goes through LLM-based processing to fix OCR errors and improve readability
- Markdown formatting (optional): reformats the corrected text into clean, consistent Markdown
- Quality assessment: compares the final output quality against the original OCR text through LLM-based evaluation
GN⁺ summary
- The LLM-Aided OCR project is a system that significantly improves the quality of OCR output using modern natural language processing techniques and large language models
- It converts PDFs to images, extracts text using Tesseract, then corrects errors with an LLM and restructures the result into Markdown format
- Supports both local and cloud-based LLMs, and optimizes performance through asynchronous processing
- The project offers various advanced features to improve the accuracy and readability of OCR output, and is especially useful for processing large documents
- Projects offering similar functionality include ABBYY FineReader and Adobe Acrobat OCR