Blog
Notes on benchmarking document AI
Guides on choosing and measuring models for document extraction: accuracy, cost and latency, on real documents.
The 99% Accuracy Trap: Why Document AI Benchmarks Lie About Real-World Performance
Benchmarks claim 97% to 99% accuracy, but production success often drops 15 to 25 points. Why it happens and what to measure instead.
Read-
OCR benchmarks compared: before and after vision-language models
From ICDAR and DocVQA to OmniDocBench, olmOCR-Bench and ExtractBench: how OCR benchmarks changed with vision-language models, and which ones to trust in 2026.
Read -
CER vs field-level accuracy: why character error rate misleads on structured extraction
Character error rate is the classic OCR metric — and the wrong one for structured field extraction. Play with a live invoice, watch CER and field-level accuracy disagree, and see why one maps to business impact and the other doesn't.
Read -
LLM OCR pricing calculator: your real cost per document
Calculate the real cost per document and per month of LLM OCR. Compare the latest GPT, Gemini, Claude and Qwen models on measured token counts.
Read -
How to choose an OCR and document-extraction model in 2026
Traditional OCR, vision LLMs, or a specialized document-AI API? A practical, vendor-neutral framework for picking the model that reads your documents — scored on accuracy, cost, and latency, not on someone else's leaderboard.
Read -
How OCR Bench scores field extraction: correct, wrong, missing, hallucinated
Diffing raw strings is a terrible way to grade an LLM, and so is a grader that guesses. Here's the processing-and-comparison engine OCR Bench uses to turn raw model output into a fair, per-field verdict: what it cleans, what it deliberately leaves alone, and why.
Read -
Introducing OCR Bench: stop rebuilding the same OCR benchmark
Every team that picks a document-extraction model builds the same throwaway benchmark: scripts, spreadsheets, hand-grading. OCR Bench is that work, done once. Bring your documents, get one scorecard for accuracy, cost, and latency.
Read