CER vs field-level accuracy: why character error rate misleads on structured extraction

If you've benchmarked OCR before, you've measured character error rate (CER). It's the field's default metric, it's easy to compute, and for the job it was designed for — transcribing pages of text — it's the right tool. But the moment your goal shifts from transcribing text to extracting fields, CER stops measuring the thing you care about. Worse, it can rank a dangerous model above a safe one.

Every number in this post comes from one six-field invoice, and you can edit it yourself further down.

This post explains what CER and WER actually measure, the specific ways they mislead on structured extraction, and why field-level accuracy is the metric you should be reporting instead.

What CER and WER measure

Character error rate is built on edit distance — the minimum number of single-character insertions, deletions, and substitutions to turn the model's output into the reference text, divided by the number of characters in the reference:

CER = (substitutions + insertions + deletions) / reference_characters

Word error rate (WER) is the same idea at word granularity. Both answer one question well: how close, character by character, is this transcription to the truth? For a scanned book or an audio transcript, that's exactly what you want.

Where it breaks on structured data

Field extraction is a different task. You're not transcribing a page — you're pulling specific values (invoice_total, due_date, vendor_name) and putting them in named slots. CER was never designed for that, and the mismatch shows up in four concrete ways.

1. It rewards "almost right" on values that must be exact

CER is a similarity score, so it gives partial credit. But a total of 1240.00 read as 1240.06 is 98.2% character-similar and 100% wrong — the payment is incorrect. For free text, "close" is good. For a field that feeds a ledger, a payment, or a legal record, there's no partial credit: the value is right or it isn't. CER's smooth gradient hides exactly the failures that cost money.

2. It can't see structure, so it can't see swaps

CER scores a transcription — a bag of characters in sequence. It has no notion of which characters were assigned to which slot. A model that puts the ship-to address in the bill-to slot has transcribed the page perfectly: every character is present and correct. Its CER is 0.0%. At the character level nothing is wrong; at the data level you just shipped an order to the billing department.

3. It punishes harmless formatting differences

The flip side: a date returned as 24/06/2026 when your ground truth says 2026-06-24 is, to CER, eight character edits — a worse score than getting the invoice total wrong — even though it's the same date. CER conflates "wrong value" with "right value, different format." Field-level evaluation handles this with per-field normalization (parse both sides to a date, then compare), so formatting noise doesn't masquerade as error.

4. It collapses distinct failure modes into one number

The single most important distinction in extraction is invisible to CER:

• A missing field (model left it blank) is an honest gap — you can flag it for review.
• A hallucinated field (model invented a plausible value not on the page) is a silent data-corruption bug — it looks like a real answer.

Deleting a seven-character true value and inserting a seven-character invented one cost the same edit distance, so CER gives them the same score. That is precisely backwards from what the business needs.

Try it: one invoice, six fields

Rather than take any of that on faith, edit the extraction yourself. Change any value below — or load one of the six scenarios — and watch the two metrics react. The CER here is computed the only way a transcription metric can compute it: over the extracted values as one page transcript, blind to which slot each value landed in.

Three of those scenarios are worth sitting with, because CER scores them identically at 14.3%: the reformatted date (all six fields correct), the blank total (an honest gap), and the invented purchase-order number (fabricated data). One is a non-event, one is a queue item, one is a production incident. CER gives you a single number that cannot tell them apart. Field-level accuracy returns three different answers.

Field-level accuracy: what to measure instead

Field-level evaluation grades each field independently against its ground-truth value, after normalization, and sorts every result into a small set of mutually exclusive outcomes.

From these you get per-field accuracy (which fields a model is reliable on), per-document accuracy (how often a whole document comes out clean), and — crucially — separate counts for the two failure modes that matter most. That's a scorecard you can actually make a decision from. We describe the full grading pipeline in how OCR Bench scores field extraction.

The ranking flips

Take the same invoice and two candidate models. Model A misreads one digit of the total. Model B returns the due date in European format and is otherwise perfect. Switch the metric and the leaderboard inverts.

Read that gap again: Model B's CER is eight times worse than Model A's, and Model B is the model you want in production. If you pick on CER, you pick the one that quietly changes a payment by six cents — or, on a different invoice, by six thousand.

When CER is still the right metric

This isn't "CER is bad." It's "CER answers a different question." The two metrics are tools for two different jobs, and the mistake is using the transcription tool to grade a database.

The takeaway

Measure the task you're doing. If the task is "fill these named fields with the right values," then character similarity is the wrong yardstick — it gives partial credit where none exists, ignores structure, penalizes formatting, and blurs the line between an honest blank and a fabricated value. Score per field, normalize before comparing, and keep missing and hallucinated separate.

That's how OCR Bench grades every benchmark. Start free, bring your documents and ground truth, and see your models ranked on the metric that actually matches your job — not the one that's easiest to compute.