Skip to content
Search

Optical character recognition

lastmod 2026-10-30created 2016-02-1245 edits

Optical character recognition (OCR) is the process of acquiring images of handwritten or machine-printed characters with an image scanner and converting them into machine-readable characters.

Software that converts images of printed type in documents, obtained by image scanning, into formats such as character codes that a computer can edit is generally called OCR, and OCR began as a research field of artificial intelligence and machine vision.

Optical character recognition using optical techniques such as mirrors and lenses, and digital character recognition using scanners and algorithms, were once considered separate areas, but the term optical character recognition is now regarded as including digital character recognition.

Early systems required "training", meaning reading samples of a specific typeface in advance in order to read that typeface, but now most typefaces can be converted with high accuracy. Some systems can produce output files in document formats such as word processor files that closely match the scanned image, and some of them correctly recognize documents even when they contain non-text parts such as images.

Categories

Basic pipeline

Ocropus-pipeline.png

Deep learning based

Projects

Cloud services

  • Google Cloud Vision
  • AWS Textract
  • Azure Document Intelligence (Form Recognizer)
  • Naver Clova
  • Upstage
  • Mistral OCR

OCR service comparison table

Comparison type

Tesseract

EasyOCR

Google Vision

AWS Textract

Azure Document Intelligence

Naver Clova

Upstage

PaddleOCR

Open Source

O

O

X

X

X

X

X

O

Korean recognition

Mid

Low

Upper-mid

Lowest (not supported)

Upper-mid

High

High

Mid (needs further testing)

English recognition

High

Low

High

High

High

High

High

High

Table + text recognition

Low

Low1

High

High (Only Eng)

High

Upper-mid

High

Mid

Speed

Mid

Lowest

Highest

Upper-mid

Lower-mid

Mid

Low

Specialized models2 provided

X

X

O

O

O (receipts, business cards, insurance/tax documents, ID cards, etc.)

O(receipts, business cards, business registration certificates, etc.)

O(receipts)

X

Model Customization3

O

O

X

X

O

O

X

O

Price (per request)4

Free

Free

$1.5 (free up to 1,000 requests per month)

-

$0.01 (pay-as-you-go: 500 pages free per month)

3 KRW (300 requests free per month)

3 KRW

Free

API difficulty

Low

Low

Upper-mid

Mid

Low

Low

Low

Mid

Customer support

X

X

X

O

O

O

O

X

Confidence score provided

O

O

△

O

O

O

O (provided separately per document and per word)

O

Overall assessment

  • A long-established, representative OCR
  • Compared with paid services it has the drawback of needing preprocessing, but it does the basics faithfully
  • Easy, intuitive usage is an advantage, but its Korean recognition rate is very poor, so it looks hard to use in practice
  • Very slow
  • Overwhelming speed.
  • Initial setup is relatively cumbersome.
  • Recognition rates vary with image quality such as shading/shadows, but it performs well for both Korean and English
  • Best for English data
  • Little flexibility, but pre-built models can handle most English documents
  • Additional verification possible through per-region entity recognition and confidence scores
  • A more advanced service than the earlier Azure Vision service
  • Provides a variety of specialized models, so further testing per use case is needed
  • From the recently released V4.0, "Model Customization" is possible, as with Naver Clova
  • Korean recognition rate is fairly good
  • Currently the most well-rounded and excellent Korean OCR service
  • Provides a variety of specialized models, and when none fits, you can "train my own data" on recurring forms through "template creation"
  • Documentation is poor, and the UI/UX is not user-friendly either
  • There are few references, but judging heuristically, its table + text recognition rate looks the best
  • Provides a variety of models
  • Better than EasyOCR in speed & performance, but compared with Tesseract it varies by test image
  • Version control needed (only compatible with Python 3.10 or lower -> downgrade needed)

Open-source OCR SoTA models (as of June 2026)

OCR is a fast-changing field, and this page summarizes the open-source State-of-the-Art models as of June 2026 by use case. Document parsing models are mainly evaluated on OmniDocBench (currently v1.7).

Document Parsing — VLM-based

This is currently the area with the fiercest SoTA competition.

Model

Parameters

Features

Notes

PaddleOCR-VL (1.6)

0.9B

OmniDocBench v1.6 SoTA (96.33)

109 languages, supports vLLM/SGLang

GLM-OCR (Z.ai)

0.9B

94.6 on v1.5, outperforms Gemini 3 Pro and GPT-5.2

vLLM/SGLang/Ollama

MinerU2.5 / Pro

1.2B

2-stage with separate layout/recognition

Strong on complex tables and formulas

dots.ocr (1.5)

—

Multilingual layout parsing with a single VLM

Image-only input

DeepSeek-OCR (2)

3B MoE

Optical compression, overwhelming token efficiency

Best for large-scale data generation

olmOCR-2 (AllenAI)

—

Trained with unit-test rewards, fully open

Bulk PDF processing

Recommendations

  • Safe first choice: PaddleOCR-VL — A lightweight 0.9B model that scored 96.33 on OmniDocBench v1.6, outperforming MinerU2.5-Pro (1.2B, 95.69), Gemini 3 Pro, and Qwen3-VL-235B.5 Combines a NaViT-style dynamic resolution visual encoder with ERNIE-4.5-0.3B, multilingual with 109 languages.6 Korean support is also strong, in keeping with the PaddleOCR line's track record.
  • Token efficiency / large-scale data pipelines: DeepSeek-OCR — Beats GOT-OCR2.0 (256 tokens/page) with only 100 vision tokens, and outperforms MinerU2.0 (an average of 6000+ tokens/page) with fewer than 800 tokens. Can generate 200,000+ pages per day on a single A100-40G.7

Traditional OCR engines — lightweight / for pipelines

When a VLM is overkill (line-level detection + recognition, low-resource environments).

  • PaddleOCR (PP-OCRv5) — Detection + recognition pipeline, multilingual including Korean, works even without a GPU
  • docTR / EasyOCR — For simple text extraction, lightweight and easy to integrate
  • OpenOCR (SVTRv2, etc.) — A toolkit for scene text recognition research and reproduction8

Caution: benchmark saturation

With the latest models reaching very high accuracy on OmniDocBench, the benchmark is effectively saturated. Score differences among top models are small, and there is still a large long tail of complex document edge cases where even the best models fail.9 So the most important thing in making a final choice is to test directly with documents from your own domain.

Why extracting data from PDFs is still hard

Limitations of OCR (optical character recognition)

  • PDF files hold important data such as scientific research and government records, but their fixed format makes them hard for machines to read and analyze
  • PDF is a format made for print layout, so it is not suited to digital analysis
  • Many PDFs contain images of information, so OCR software is needed to convert them into data
  • OCR performance drops even further for old or handwritten documents

The unstructured data problem

  • About 80–90% of the world's organizational data is stored as unstructured data, often in PDFs
  • Extracting data is especially hard from two-column layouts, tables, charts, and low-quality scans
  • This causes major problems especially in scientific research, historical document preservation, customer service, and making technical literature accessible to AI systems

Impact by sector

  • Affects the operation of public institutions such as government records, courts, police, and social services
  • Information-dependent industries such as insurance and banking spend time and resources converting PDF data

History of OCR technology

  • In the 1970s, Ray Kurzweil developed a commercial OCR system based on pattern-matching algorithms
  • The Kurzweil Reading Machine provided character recognition for blind people
  • Traditional OCR systems work by recognizing patterns of light and dark and converting them into characters
  • Performance degrades with complex fonts, multi-column layouts, tables, and so on
  • Traditional OCR errors are predictable and thus easy to fix, but it has limitations

The rise of AI-based OCR

  • Multimodal LLMs (large language models) extract data by integrating images and text
  • Models from OpenAI, Google, Meta, and others can recognize both the visual elements of a document and the textual context at the same time
  • Traditional OCR uses character-level pattern matching, whereas AI processes documents by recognizing their layout and context
  • Amazon's Textract uses the traditional OCR approach, but LLMs can analyze documents in a broader context
  • They handle complex layouts, tables, captions, and so on better

New LLM-based OCR attempts

  • French AI company Mistral released Mistral OCR, an LLM-based document processing API
  • It aims to extract text and images from documents with complex layouts
  • Performance problems: failed to process tables in old documents and produced number errors
  • Problems with handwriting recognition → the AI generated arbitrary content (hallucination)
  • Google's Gemini 2.0 currently delivers the best performance → few errors even on complex documents

Problems with LLM-based OCR

  • Because LLMs are probabilistic models, errors are quite likely
  • Lines get dropped when a document layout repeats
  • LLMs can fail to distinguish user prompts from document content, leading to misinterpretation
  • Mismatched values in tables cause critical errors → major problems in finance, law, and medicine
  • Arbitrary text generation → human review needed

Challenges ahead

  • No perfect OCR solution exists yet
  • Google, OpenAI, and others are improving performance through context-aware AI products
  • AI companies hope to obtain AI training data by extracting data from PDFs
  • If AI can handle PDF data perfectly, it could open a new era of data analysis

See also

Favorite site

Guide

Article

References

  1. A separate paid Enterprise version exists ↩

  2. OCR models trained on specific documents such as receipts and business registration certificates; the image information of that form is usually returned in a predefined JSON structure ↩

  3. Think of it as fine-tuning. A feature that lets you fine-tune the model via "Train with My Own Data" ↩

  4. Based on 1 request, general OCR, Base Plan; varies greatly by volume/purpose (many services offer a certain number of requests for free) ↩

  5. PaddleOCR-VL-1.6 SoTA announcement post ↩

  6. PaddleOCR-VL: Boosting Multilingual Document Parsing (arXiv:2510.14528) ↩

  7. DeepSeek-OCR: Contexts Optical Compression (arXiv:2510.18234) ↩

  8. OpenOCR GitHub ↩

  9. OmniDocBench is Saturated, What's Next for OCR Benchmarks? (LlamaIndex) ↩

  10. Building_Custom_Deep_Learning_Based_OCR_models.pdf ↩

author Zer0 sitemap