OCR mini-bench

Business-first OCR benchmark for standard operational documents.

This benchmark compares OCR extraction performance on real business documents across repeated runs, so you can see both quality and consistency, not just a single score. It explicitly measures how well models transform an input document and expected keys into the correct output values. It highlights what matters in production: critical-field success, reliability over repeats (pass^n), latency, stability, and cost per successful outcome.

Leaderboard

View On Github Blog

42 docs • 16 models • 6,720 runs

Last updated: 18/03/2026

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Gemini 3 Flash Google • BALANCED	73.8%	73.8%	73.8%	0.67¢	16.0s	96.7%	95.3%	0.46¢
2	Claude Sonnet 4.6 Anthropic • BALANCED	73.8%	73.8%	73.8%	3.61¢	18.9s	94.3%	94.7%	2.46¢
3	Claude Opus 4.6 Anthropic • SOTA	73.0%	72.2%	71.8%	6.25¢	18.9s	95.8%	94.8%	4.15¢
4	Gemini 3.1 Pro Google • SOTA	68.7%	68.7%	68.7%	2.55¢	65.3s	91.5%	89.5%	1.63¢
5	Gemini 3.1 Flash-Lite Google • BALANCED	61.2%	61.2%	61.2%	0.32¢	12.8s	93.3%	93.2%	0.19¢
6	Gemini 2.5 Flash-Lite Google • BUDGET	58.6%	58.6%	58.6%	0.10¢	14.4s	94.1%	91.9%	0.06¢
7	Medium Mistral • BALANCED	54.1%	49.8%	47.5%	0.69¢	21.0s	91.0%	88.4%	0.29¢
8	Large Mistral • SOTA	50.5%	48.4%	47.3%	0.31¢	23.2s	92.0%	89.9%	0.28¢
9	OCR Mistral • SOTA	48.4%	43.5%	41.6%	0.67¢	11.8s	92.3%	91.8%	0.30¢
10	Small Mistral • BUDGET	46.2%	43.1%	41.9%	0.12¢	12.6s	88.6%	88.0%	0.05¢
11	GPT-5 OpenAI • SOTA	44.6%	39.3%	37.9%	24.20¢	19.8s	88.8%	89.4%	1.01¢
12	GPT-5.4 mini OpenAI • BALANCED	43.2%	35.9%	32.4%	3.30¢	13.7s	91.5%	92.0%	0.65¢
13	GPT-5 mini OpenAI • BALANCED	39.3%	32.7%	30.5%	3.09¢	25.0s	90.2%	89.9%	0.28¢
14	Claude Haiku 4.5 Anthropic • BUDGET	34.9%	34.9%	34.9%	3.73¢	13.6s	89.9%	89.9%	0.97¢
15	GPT-5.4 nano OpenAI • BUDGET	23.6%	13.7%	11.3%	7.05¢	19.9s	82.8%	78.7%	0.27¢
16	GPT-5 nano OpenAI • BUDGET	8.7%	5.2%	4.1%	2.88¢	17.0s	63.1%	52.6%	0.05¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 10 runs per model.

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Claude Opus 4.6 Anthropic • SOTA	84.6%	84.6%	84.6%	3.50¢	11.3s	97.8%	95.8%	2.97¢
2	Gemini 3 Flash Google • BALANCED	76.9%	76.9%	76.9%	0.30¢	7.1s	96.5%	92.2%	0.23¢
3	Claude Sonnet 4.6 Anthropic • BALANCED	76.9%	76.9%	76.9%	2.30¢	10.8s	94.1%	95.7%	1.77¢
4	GPT-5 mini OpenAI • BALANCED	63.1%	57.2%	53.9%	0.29¢	13.7s	92.0%	87.6%	0.19¢
5	Gemini 3.1 Flash-Lite Google • BALANCED	61.5%	61.5%	61.5%	0.20¢	6.4s	90.5%	89.1%	0.13¢
6	Gemini 3.1 Pro Google • SOTA	61.5%	61.5%	61.5%	1.92¢	42.8s	89.7%	86.5%	1.18¢
7	Large Mistral • SOTA	60.8%	56.1%	54.5%	0.29¢	12.6s	93.5%	87.9%	0.18¢
8	GPT-5 OpenAI • SOTA	56.1%	49.0%	46.8%	1.15¢	12.3s	91.3%	88.5%	0.65¢
9	GPT-5.4 mini OpenAI • BALANCED	55.4%	51.5%	50.0%	0.87¢	8.5s	90.0%	90.1%	0.48¢
10	Gemini 2.5 Flash-Lite Google • BUDGET	53.9%	53.9%	53.9%	0.06¢	6.5s	92.3%	86.8%	0.03¢
11	Claude Haiku 4.5 Anthropic • BUDGET	53.9%	53.9%	53.9%	1.24¢	8.0s	90.0%	86.1%	0.67¢
12	Medium Mistral • BALANCED	53.9%	53.9%	53.9%	0.34¢	10.8s	88.9%	82.6%	0.19¢
13	OCR Mistral • SOTA	52.3%	47.4%	46.3%	0.57¢	6.0s	90.8%	89.9%	0.30¢
14	GPT-5.4 nano OpenAI • BUDGET	43.1%	28.5%	24.9%	0.45¢	11.4s	87.1%	76.6%	0.19¢
15	Small Mistral • BUDGET	43.1%	34.9%	32.5%	0.07¢	5.5s	83.5%	85.1%	0.03¢
16	GPT-5 nano OpenAI • BUDGET	25.4%	15.6%	12.2%	0.14¢	11.4s	75.5%	52.7%	0.04¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 3 runs per model.

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Gemini 3 Flash Google • BALANCED	91.7%	91.7%	91.7%	0.60¢	20.7s	99.4%	99.7%	0.55¢
2	Claude Sonnet 4.6 Anthropic • BALANCED	91.7%	91.7%	91.7%	2.83¢	20.7s	97.0%	94.2%	2.59¢
3	Gemini 3.1 Pro Google • SOTA	91.7%	91.7%	91.7%	1.70¢	55.9s	95.8%	91.9%	1.56¢
4	Large Mistral • SOTA	90.8%	89.2%	87.5%	0.32¢	25.9s	96.4%	93.5%	0.29¢
5	Claude Opus 4.6 Anthropic • SOTA	83.3%	83.3%	83.3%	5.29¢	20.9s	96.7%	94.1%	4.41¢
6	Medium Mistral • BALANCED	80.8%	71.0%	66.7%	0.38¢	25.0s	95.7%	93.4%	0.31¢
7	GPT-5 OpenAI • SOTA	75.8%	68.8%	66.9%	1.50¢	23.7s	97.4%	97.3%	1.14¢
8	Gemini 3.1 Flash-Lite Google • BALANCED	75.0%	75.0%	75.0%	0.29¢	16.2s	98.2%	99.0%	0.22¢
9	Gemini 2.5 Flash-Lite Google • BUDGET	75.0%	75.0%	75.0%	0.09¢	20.4s	97.6%	98.6%	0.07¢
10	Small Mistral • BUDGET	66.7%	66.7%	66.7%	0.08¢	15.5s	93.4%	92.7%	0.05¢
11	GPT-5.4 mini OpenAI • BALANCED	64.2%	50.4%	41.2%	1.04¢	17.3s	96.7%	97.7%	0.67¢
12	OCR Mistral • SOTA	61.7%	55.3%	52.0%	0.49¢	13.5s	96.7%	98.0%	0.30¢
13	GPT-5 mini OpenAI • BALANCED	50.8%	40.6%	37.6%	0.62¢	32.1s	95.8%	96.6%	0.31¢
14	Claude Haiku 4.5 Anthropic • BUDGET	33.3%	33.3%	33.3%	3.06¢	15.9s	94.0%	95.2%	1.02¢
15	GPT-5.4 nano OpenAI • BUDGET	25.8%	12.6%	8.9%	1.07¢	24.3s	89.5%	90.8%	0.28¢
16	GPT-5 nano OpenAI • BUDGET	0.8%	0.0%	0.0%	5.62¢	18.8s	54.9%	57.4%	0.05¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 3 runs per model.

#	Model	Success	pass^3	pass^5	Cost/success	Latency	Critical	All fields	Cost/doc
1	Gemini 3 Flash Google • BALANCED	52.9%	52.9%	52.9%	1.11¢	20.1s	94.2%	93.9%	0.59¢
2	Claude Sonnet 4.6 Anthropic • BALANCED	52.9%	52.9%	52.9%	5.70¢	25.2s	91.6%	94.2%	3.02¢
3	Gemini 3.1 Pro Google • SOTA	52.9%	52.9%	52.9%	4.05¢	97.3s	89.0%	90.2%	2.14¢
4	Claude Opus 4.6 Anthropic • SOTA	51.2%	48.8%	47.5%	9.94¢	24.6s	92.8%	94.6%	5.09¢
5	Gemini 2.5 Flash-Lite Google • BUDGET	47.1%	47.1%	47.1%	0.17¢	16.2s	92.3%	90.3%	0.08¢
6	Gemini 3.1 Flash-Lite Google • BALANCED	47.1%	47.1%	47.1%	0.47¢	15.8s	91.3%	91.4%	0.22¢
7	OCR Mistral • SOTA	31.2%	27.8%	26.5%	0.96¢	15.9s	89.3%	87.6%	0.30¢
8	Small Mistral • BUDGET	28.8%	27.6%	26.5%	0.22¢	16.7s	89.0%	86.3%	0.06¢
9	Medium Mistral • BALANCED	27.6%	24.5%	21.9%	1.36¢	27.1s	88.5%	89.2%	0.38¢
10	Claude Haiku 4.5 Anthropic • BUDGET	17.6%	17.6%	17.6%	6.90¢	16.8s	85.6%	88.3%	1.22¢
11	GPT-5.4 mini OpenAI • BALANCED	10.0%	5.9%	5.9%	7.99¢	15.3s	87.8%	88.1%	0.80¢
12	GPT-5 mini OpenAI • BALANCED	4.1%	0.3%	0.0%	8.36¢	29.2s	82.9%	85.6%	0.34¢
13	GPT-5 OpenAI • SOTA	1.8%	0.1%	0.0%	69.95¢	23.3s	77.8%	82.5%	1.23¢
14	GPT-5.4 nano OpenAI • BUDGET	1.8%	0.0%	0.0%	19.65¢	23.9s	71.9%	68.6%	0.35¢
15	Large Mistral • SOTA	0.0%	0.0%	0.0%	n/a	31.0s	86.1%	88.5%	0.36¢
16	GPT-5 nano OpenAI • BUDGET	0.0%	0.0%	0.0%	n/a	20.7s	58.9%	47.7%	0.06¢

^† passⁿ metric: Probability of n consecutive successes in n runs (strict).

^† Variance column: Shows min–max interval with bar width indicating spread.

^† All metrics: Aggregated across all documents and 4 runs per model.