OCR mini-bench

Business-first OCR benchmark for standard operational documents.

This benchmark compares OCR extraction performance on real business documents across repeated runs, so you can see both quality and consistency, not just a single score. It explicitly measures how well models transform an input document and expected keys into the correct output values. It highlights what matters in production: critical-field success, reliability over repeats (pass^n), latency, stability, and cost per successful outcome. Every model is run at its lowest thinking mode for a level playing field, unless noted otherwise (*).

Leaderboard

View On Github Blog

42 docs • 31 models • 13,020 runs

Last updated: 22/07/2026

Thinking variants Off

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Gemini 3 Flash Google • BALANCED	73.8%	73.8%	73.8%	0.67¢	16.0s	96.7%	95.3%	0.46¢
2	Claude Sonnet 4.6 Anthropic • BALANCED	73.8%	73.8%	73.8%	3.61¢	18.9s	94.3%	94.7%	2.46¢
3	Gemini 3.6 Flash medium thinking Google • BALANCED	73.8%	66.7%	62.9%	1.82¢	28.0s	95.4%	92.7%	1.29¢
4	Claude Opus 4.6 Anthropic • SOTA	73.0%	72.2%	71.8%	6.25¢	18.9s	95.8%	94.8%	4.15¢
5	GPT-5.6 Sol OpenAI • SOTA	71.5%	64.3%	62.3%	8.28¢	20.7s	96.3%	96.0%	5.97¢
6	Gemini 3.1 Pro Google • SOTA	68.7%	68.7%	68.7%	2.55¢	65.3s	91.5%	89.5%	1.63¢
7	Claude Opus 5 Anthropic • SOTA	68.6%	65.1%	64.4%	5.98¢	17.4s	95.9%	94.7%	4.09¢
8	Claude Opus 4.7 Anthropic • SOTA	66.1%	63.0%	62.4%	6.59¢	18.7s	95.2%	94.6%	4.13¢
9	Gemini 3.6 Flash Google • BALANCED	65.9%	60.0%	58.4%	2.31¢	13.7s	95.7%	93.8%	1.39¢
10	Claude Sonnet 5 Anthropic • BALANCED	64.6%	56.4%	53.8%	4.96¢	18.2s	94.3%	93.1%	3.09¢
11	Gemini 3.5 Flash-Lite Google • BUDGET	63.4%	53.9%	49.7%	0.69¢	10.6s	95.1%	92.7%	0.42¢
12	GPT-5.6 Terra OpenAI • SOTA	62.8%	55.3%	52.1%	4.95¢	14.6s	95.5%	95.0%	3.00¢
13	GPT-5.6 Luna OpenAI • BALANCED	62.8%	52.9%	49.9%	1.90¢	13.6s	95.0%	94.3%	1.20¢
14	GPT-5.6 Luna medium thinking OpenAI • BALANCED	61.9%	52.4%	48.7%	2.16¢	15.0s	94.9%	94.3%	1.34¢
15	Gemini 3.1 Flash-Lite Google • BALANCED	61.2%	61.2%	61.2%	0.32¢	12.8s	93.3%	93.2%	0.19¢
16	GPT-5.6 Terra medium thinking OpenAI • SOTA	60.4%	54.6%	51.2%	5.23¢	13.4s	95.0%	94.8%	3.07¢
17	Gemini 2.5 Flash-Lite Google • BUDGET	58.6%	58.6%	58.6%	0.10¢	14.4s	94.1%	91.9%	0.06¢
18	GPT-5.5 OpenAI • SOTA	57.3%	47.3%	43.2%	9.23¢	16.2s	94.2%	94.0%	4.13¢
19	Gemini 3.5 Flash Google • BALANCED	57.3%	57.3%	57.3%	3.36¢	13.8s	94.8%	93.9%	1.67¢
20	Medium Mistral • BALANCED	54.1%	49.8%	47.5%	0.69¢	21.0s	91.0%	88.4%	0.29¢
21	Gemini 3.5 Flash-Lite high thinking Google • BUDGET	52.9%	43.5%	40.2%	0.84¢	20.3s	93.5%	91.6%	0.42¢
22	Large Mistral • SOTA	50.5%	48.4%	47.3%	0.31¢	23.2s	92.0%	89.9%	0.28¢
23	GPT-5.4 OpenAI • SOTA	49.2%	38.8%	35.9%	7.82¢	26.9s	92.7%	93.2%	2.22¢
24	OCR Mistral • SOTA	48.4%	43.5%	41.6%	0.67¢	11.8s	92.3%	91.8%	0.30¢
25	Small Mistral • BUDGET	46.2%	43.1%	41.9%	0.12¢	12.6s	88.6%	88.0%	0.05¢
26	GPT-5 OpenAI • SOTA	44.6%	39.3%	37.9%	24.20¢	19.8s	88.8%	89.4%	1.01¢
27	GPT-5.4 mini OpenAI • BALANCED	43.2%	35.9%	32.4%	3.30¢	13.7s	91.5%	92.0%	0.65¢
28	GPT-5 mini OpenAI • BALANCED	39.3%	32.7%	30.5%	3.09¢	25.0s	90.2%	89.9%	0.28¢
29	Claude Haiku 4.5 Anthropic • BUDGET	34.9%	34.9%	34.9%	3.73¢	13.6s	89.9%	89.9%	0.97¢
30	GPT-5.4 nano OpenAI • BUDGET	23.6%	13.7%	11.3%	7.05¢	19.9s	82.8%	78.7%	0.27¢
31	GPT-5 nano OpenAI • BUDGET	8.7%	5.2%	4.1%	2.88¢	17.0s	63.1%	52.6%	0.05¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 10 runs per model.

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Claude Opus 4.6 Anthropic • SOTA	84.6%	84.6%	84.6%	3.50¢	11.3s	97.8%	95.8%	2.97¢
2	Gemini 3 Flash Google • BALANCED	76.9%	76.9%	76.9%	0.30¢	7.1s	96.5%	92.2%	0.23¢
3	Claude Sonnet 4.6 Anthropic • BALANCED	76.9%	76.9%	76.9%	2.30¢	10.8s	94.1%	95.7%	1.77¢
4	GPT-5.5 OpenAI • SOTA	71.5%	62.8%	58.5%	4.30¢	10.3s	94.8%	92.6%	3.08¢
5	Gemini 3.6 Flash medium thinking Google • BALANCED	69.2%	58.8%	53.4%	1.20¢	15.4s	95.4%	93.5%	0.83¢
6	Claude Opus 5 Anthropic • SOTA	65.4%	62.2%	61.6%	4.25¢	9.7s	94.9%	93.8%	2.78¢
7	GPT-5.6 Terra OpenAI • SOTA	64.6%	55.5%	51.7%	2.68¢	9.7s	94.4%	93.9%	1.73¢
8	Claude Sonnet 5 Anthropic • BALANCED	63.9%	54.1%	50.4%	3.30¢	10.0s	93.5%	94.4%	2.11¢
9	GPT-5 mini OpenAI • BALANCED	63.1%	57.2%	53.9%	0.29¢	13.7s	92.0%	87.6%	0.19¢
10	Gemini 3.5 Flash Google • BALANCED	61.5%	61.5%	61.5%	1.82¢	7.8s	94.3%	90.9%	1.12¢
11	Gemini 3.6 Flash Google • BALANCED	61.5%	49.4%	46.5%	1.33¢	7.9s	94.3%	90.5%	0.82¢
12	Gemini 3.5 Flash-Lite high thinking Google • BUDGET	61.5%	54.5%	53.9%	0.46¢	11.5s	93.9%	91.7%	0.28¢
13	Gemini 3.1 Flash-Lite Google • BALANCED	61.5%	61.5%	61.5%	0.20¢	6.4s	90.5%	89.1%	0.13¢
14	Gemini 3.1 Pro Google • SOTA	61.5%	61.5%	61.5%	1.92¢	42.8s	89.7%	86.5%	1.18¢
15	GPT-5.6 Sol OpenAI • SOTA	60.8%	54.2%	53.9%	5.55¢	12.5s	94.0%	94.7%	3.37¢
16	Large Mistral • SOTA	60.8%	56.1%	54.5%	0.29¢	12.6s	93.5%	87.9%	0.18¢
17	GPT-5.4 OpenAI • SOTA	58.5%	52.8%	50.2%	2.64¢	19.7s	92.0%	91.6%	1.55¢
18	Claude Opus 4.7 Anthropic • SOTA	57.7%	53.9%	53.9%	4.81¢	9.4s	93.3%	93.7%	2.78¢
19	GPT-5.6 Terra medium thinking OpenAI • SOTA	56.9%	50.0%	47.9%	3.17¢	8.5s	93.1%	93.6%	1.81¢
20	GPT-5 OpenAI • SOTA	56.1%	49.0%	46.8%	1.15¢	12.3s	91.3%	88.5%	0.65¢
21	GPT-5.4 mini OpenAI • BALANCED	55.4%	51.5%	50.0%	0.87¢	8.5s	90.0%	90.1%	0.48¢
22	GPT-5.6 Luna medium thinking OpenAI • BALANCED	53.9%	46.5%	46.1%	1.60¢	10.5s	93.1%	91.7%	0.86¢
23	Gemini 2.5 Flash-Lite Google • BUDGET	53.9%	53.9%	53.9%	0.06¢	6.5s	92.3%	86.8%	0.03¢
24	Claude Haiku 4.5 Anthropic • BUDGET	53.9%	53.9%	53.9%	1.24¢	8.0s	90.0%	86.1%	0.67¢
25	Medium Mistral • BALANCED	53.9%	53.9%	53.9%	0.34¢	10.8s	88.9%	82.6%	0.19¢
26	Gemini 3.5 Flash-Lite Google • BUDGET	53.1%	43.6%	39.8%	0.49¢	6.0s	93.1%	88.8%	0.26¢
27	GPT-5.6 Luna OpenAI • BALANCED	53.1%	47.4%	46.3%	1.34¢	9.2s	92.5%	91.7%	0.71¢
28	OCR Mistral • SOTA	52.3%	47.4%	46.3%	0.57¢	6.0s	90.8%	89.9%	0.30¢
29	GPT-5.4 nano OpenAI • BUDGET	43.1%	28.5%	24.9%	0.45¢	11.4s	87.1%	76.6%	0.19¢
30	Small Mistral • BUDGET	43.1%	34.9%	32.5%	0.07¢	5.5s	83.5%	85.1%	0.03¢
31	GPT-5 nano OpenAI • BUDGET	25.4%	15.6%	12.2%	0.14¢	11.4s	75.5%	52.7%	0.04¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 3 runs per model.

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Gemini 3 Flash Google • BALANCED	91.7%	91.7%	91.7%	0.60¢	20.7s	99.4%	99.7%	0.55¢
2	Gemini 3.6 Flash medium thinking Google • BALANCED	91.7%	91.7%	91.7%	1.43¢	24.8s	97.7%	93.2%	1.31¢
3	Claude Sonnet 4.6 Anthropic • BALANCED	91.7%	91.7%	91.7%	2.83¢	20.7s	97.0%	94.2%	2.59¢
4	Gemini 3.1 Pro Google • SOTA	91.7%	91.7%	91.7%	1.70¢	55.9s	95.8%	91.9%	1.56¢
5	Gemini 3.6 Flash Google • BALANCED	90.8%	89.2%	87.5%	1.81¢	17.0s	99.3%	99.2%	1.65¢
6	Large Mistral • SOTA	90.8%	89.2%	87.5%	0.32¢	25.9s	96.4%	93.5%	0.29¢
7	Claude Opus 4.7 Anthropic • SOTA	88.3%	84.0%	83.4%	4.91¢	20.9s	98.3%	96.0%	4.34¢
8	GPT-5.6 Sol OpenAI • SOTA	86.7%	77.2%	75.2%	8.17¢	21.6s	99.0%	99.3%	7.08¢
9	Gemini 3.5 Flash-Lite Google • BUDGET	85.8%	77.8%	72.9%	0.57¢	13.3s	98.8%	99.4%	0.49¢
10	Claude Opus 4.6 Anthropic • SOTA	83.3%	83.3%	83.3%	5.29¢	20.9s	96.7%	94.1%	4.41¢
11	Medium Mistral • BALANCED	80.8%	71.0%	66.7%	0.38¢	25.0s	95.7%	93.4%	0.31¢
12	GPT-5.6 Luna OpenAI • BALANCED	75.8%	68.3%	66.9%	1.86¢	15.0s	98.3%	98.3%	1.41¢
13	GPT-5 OpenAI • SOTA	75.8%	68.8%	66.9%	1.50¢	23.7s	97.4%	97.3%	1.14¢
14	Gemini 3.5 Flash Google • BALANCED	75.0%	75.0%	75.0%	2.43¢	17.0s	98.2%	99.3%	1.82¢
15	Gemini 3.1 Flash-Lite Google • BALANCED	75.0%	75.0%	75.0%	0.29¢	16.2s	98.2%	99.0%	0.22¢
16	GPT-5.6 Terra medium thinking OpenAI • SOTA	75.0%	72.5%	70.8%	4.81¢	15.1s	97.7%	98.4%	3.61¢
17	Gemini 2.5 Flash-Lite Google • BUDGET	75.0%	75.0%	75.0%	0.09¢	20.4s	97.6%	98.6%	0.07¢
18	Claude Opus 5 Anthropic • SOTA	75.0%	68.3%	66.9%	5.72¢	19.6s	97.5%	96.3%	4.29¢
19	GPT-5.6 Luna medium thinking OpenAI • BALANCED	74.2%	65.6%	62.7%	2.03¢	16.1s	98.2%	98.8%	1.51¢
20	Claude Sonnet 5 Anthropic • BALANCED	74.2%	68.1%	66.9%	3.83¢	18.6s	95.8%	92.7%	2.84¢
21	GPT-5.4 OpenAI • SOTA	73.3%	58.2%	54.4%	3.14¢	32.7s	97.8%	98.3%	2.30¢
22	GPT-5.6 Terra OpenAI • SOTA	72.5%	67.4%	66.7%	4.84¢	16.0s	97.7%	98.4%	3.51¢
23	GPT-5.5 OpenAI • SOTA	71.7%	58.7%	54.9%	5.99¢	19.5s	98.0%	98.0%	4.29¢
24	Small Mistral • BUDGET	66.7%	66.7%	66.7%	0.08¢	15.5s	93.4%	92.7%	0.05¢
25	GPT-5.4 mini OpenAI • BALANCED	64.2%	50.4%	41.2%	1.04¢	17.3s	96.7%	97.7%	0.67¢
26	OCR Mistral • SOTA	61.7%	55.3%	52.0%	0.49¢	13.5s	96.7%	98.0%	0.30¢
27	Gemini 3.5 Flash-Lite high thinking Google • BUDGET	52.5%	45.1%	41.7%	0.83¢	19.3s	94.5%	92.3%	0.44¢
28	GPT-5 mini OpenAI • BALANCED	50.8%	40.6%	37.6%	0.62¢	32.1s	95.8%	96.6%	0.31¢
29	Claude Haiku 4.5 Anthropic • BUDGET	33.3%	33.3%	33.3%	3.06¢	15.9s	94.0%	95.2%	1.02¢
30	GPT-5.4 nano OpenAI • BUDGET	25.8%	12.6%	8.9%	1.07¢	24.3s	89.5%	90.8%	0.28¢
31	GPT-5 nano OpenAI • BUDGET	0.8%	0.0%	0.0%	5.62¢	18.8s	54.9%	57.4%	0.05¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 3 runs per model.

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	GPT-5.6 Sol OpenAI • SOTA	67.1%	61.5%	57.7%	11.13¢	28.0s	95.9%	94.0%	7.46¢
2	Claude Opus 5 Anthropic • SOTA	65.3%	64.7%	64.7%	7.97¢	23.1s	95.4%	94.0%	5.21¢
3	Gemini 3.6 Flash medium thinking Google • BALANCED	60.6%	49.7%	43.5%	2.83¢	43.7s	93.2%	91.4%	1.72¢
4	GPT-5.6 Luna OpenAI • BALANCED	59.4%	42.9%	36.4%	2.50¢	16.5s	94.3%	92.9%	1.49¢
5	GPT-5.6 Luna medium thinking OpenAI • BALANCED	57.6%	45.0%	37.1%	2.84¢	18.4s	93.3%	92.6%	1.64¢
6	Claude Sonnet 5 Anthropic • BALANCED	55.9%	47.0%	44.3%	7.73¢	26.0s	93.5%	92.2%	4.32¢
7	Gemini 3 Flash Google • BALANCED	52.9%	52.9%	52.9%	1.11¢	20.1s	94.2%	93.9%	0.59¢
8	Claude Sonnet 4.6 Anthropic • BALANCED	52.9%	52.9%	52.9%	5.70¢	25.2s	91.6%	94.2%	3.02¢
9	Gemini 3.1 Pro Google • SOTA	52.9%	52.9%	52.9%	4.05¢	97.3s	89.0%	90.2%	2.14¢
10	Claude Opus 4.7 Anthropic • SOTA	52.4%	51.2%	50.0%	10.05¢	25.9s	94.1%	94.1%	5.26¢
11	GPT-5.6 Terra OpenAI • SOTA	51.2%	43.2%	37.9%	7.32¢	18.1s	94.5%	92.6%	3.74¢
12	Gemini 3.5 Flash-Lite Google • BUDGET	51.2%	40.2%	36.4%	1.00¢	12.6s	93.3%	89.9%	0.51¢
13	Claude Opus 4.6 Anthropic • SOTA	51.2%	48.8%	47.5%	9.94¢	24.6s	92.8%	94.6%	5.09¢
14	GPT-5.6 Terra medium thinking OpenAI • SOTA	49.4%	41.4%	35.0%	7.71¢	16.5s	94.1%	92.6%	3.81¢
15	Gemini 2.5 Flash-Lite Google • BUDGET	47.1%	47.1%	47.1%	0.17¢	16.2s	92.3%	90.3%	0.08¢
16	Gemini 3.1 Flash-Lite Google • BALANCED	47.1%	47.1%	47.1%	0.47¢	15.8s	91.3%	91.4%	0.22¢
17	Gemini 3.6 Flash Google • BALANCED	45.3%	41.4%	41.2%	3.80¢	16.2s	93.4%	91.7%	1.72¢
18	Gemini 3.5 Flash-Lite high thinking Google • BUDGET	44.7%	31.1%	25.1%	1.22¢	30.1s	92.0%	90.7%	0.55¢
19	Gemini 3.5 Flash Google • BALANCED	35.3%	35.3%	35.3%	5.84¢	16.5s	91.8%	91.4%	2.06¢
20	OCR Mistral • SOTA	31.2%	27.8%	26.5%	0.96¢	15.9s	89.3%	87.6%	0.30¢
21	GPT-5.5 OpenAI • SOTA	28.8%	20.6%	16.3%	17.40¢	18.7s	89.8%	91.5%	5.02¢
22	Small Mistral • BUDGET	28.8%	27.6%	26.5%	0.22¢	16.7s	89.0%	86.3%	0.06¢
23	Medium Mistral • BALANCED	27.6%	24.5%	21.9%	1.36¢	27.1s	88.5%	89.2%	0.38¢
24	Claude Haiku 4.5 Anthropic • BUDGET	17.6%	17.6%	17.6%	6.90¢	16.8s	85.6%	88.3%	1.22¢
25	GPT-5.4 OpenAI • SOTA	15.9%	5.5%	3.1%	17.69¢	28.2s	88.3%	89.7%	2.81¢
26	GPT-5.4 mini OpenAI • BALANCED	10.0%	5.9%	5.9%	7.99¢	15.3s	87.8%	88.1%	0.80¢
27	GPT-5 mini OpenAI • BALANCED	4.1%	0.3%	0.0%	8.36¢	29.2s	82.9%	85.6%	0.34¢
28	GPT-5 OpenAI • SOTA	1.8%	0.1%	0.0%	69.95¢	23.3s	77.8%	82.5%	1.23¢
29	GPT-5.4 nano OpenAI • BUDGET	1.8%	0.0%	0.0%	19.65¢	23.9s	71.9%	68.6%	0.35¢
30	Large Mistral • SOTA	0.0%	0.0%	0.0%	n/a	31.0s	86.1%	88.5%	0.36¢
31	GPT-5 nano OpenAI • BUDGET	0.0%	0.0%	0.0%	n/a	20.7s	58.9%	47.7%	0.06¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 4 runs per model.

(*) Thinking mode. Every model is run at its lowest thinking mode — this establishes the lower boundary of capability and keeps a level playing field, since default thinking budgets differ across models and providers. So: lowest thinking, unless noted otherwise. From 22/07 we also started testing models at their default / higher thinking modes to see how they compare against their non-thinking counterparts; those runs are tagged with a thinking badge and can be shown with the Thinking variants toggle above the table.