o4-mini

+ contribute

o4-mini is a closed-weights language model from OpenAI, released April 2025.

66.3±0.6#125 of 719 LLMsfrom 54 benchmark results across 18 sources

$ how it compares

The LLMs ranked either side of o4-mini, out of 828 with a score, then more from OpenAI.

#	model	score	where	RAM	released
142	Qwen3 235B-A22B Thinking 2507 Alibaba	66±1	api $0.97/M +local	~76GB	2025-07
143	Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)	66±1	api $0.28/M		2025-11
144	o4-mini OpenAI	66±1	api $1.93/M		2025-04
145	Qwen3.6 Flash Alibaba	66±1	api $0.56/M		2026-04
146	MiMo-V2-Flash (309B-A15B) Xiaomi	66±1	local	~108GB	2025-12
	more from OpenAI
1	GPT-6 Astra OpenAI	87±1	api $20.00/M		2026-09
3	GPT-6.1 Sol OpenAI	85±1	api $4.00/M		2026-09
7	GPT-6 Sol OpenAI	83±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #150 of 300 · 17% of its score · 6 tests

Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
τ²-bench Telecom lab +2	40.5% ✎	▃·▅▅▃▃··█▃▃▅ beats 3/13	very hard · avg 8.1%	▇▇▇ 8.0%	sharp · at its level · several runs
τ²-bench Retail +1	68.3% ✎	▁····▂▁▃▆█▃▁ beats 27/44	medium · avg 41%	▇ 2.8%	at its level · several runs
Finance Agent Benchmark v1 (537 questions, class-balanced accuracy)	37.3% ✎	█·▃▃·█▅▅▅▅▃▅ beats 16/18	very hard · avg 5.8%	▇ 2.6%	sharp · at its level
τ²-bench Airline +1	52.1% ✎	▁▁·▂▁▃▂▁█▅▂▃ beats 28/42	hard · avg 29%	▇ 2.1%	at its level · several runs
BFCL V4 (December 2025 archive)	53.2% ✎	▁▁▁▂██▅▅▆▂▂▂ beats 61/77	hard · avg 32%	▇ 1.5%	at its level
Galileo Agent Leaderboard v1 (tool selection quality)	88.9% ✎	▃▃▃▃▆▅▆▅█▂▅▂ beats 20/31	easy · avg 79%	▇ 0.3%	blunt

coding▁▁▁▃▆██▇▆▄▃▁ #93 of 458 · 13% of its score · 5 tests

Beats Grok 4.20 (0309 reasoning), Grok 4.3, Muse Glimmer 30B here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Aider polyglot (225 exercises, two attempts)	72.0% ✎	▂·▂▃▅▂▃█▆▄▂▂ beats 41/49	very hard · avg 8.5%	▇▇▇ 7.9%	sharp · at its level
LiveCodeBench official (Aug 1, 2024–May 1, 2025)	80.2% ✎	▃▃▅▃··▃▅█▅▅▃ beats 15/15	hard · avg 25%	▇ 2.2%	at its level
WebApp1K-Duo-React pass@1 (onekq-ai)	72.6% ✎	▁··▁·▁▁▁▆█▇█ beats 34/45	hard · avg 32%	▇ 1.4%	at its level
LiveCodeBench v6 (Feb–May 2025)	71.8% ✎	▁▁▃▁█▅▄▇▅▅▃█ beats 22/38	medium · avg 36%	▇ 1.0%	at its level
HumanEval (simple-evals) lab	99.3% ✎	▆█▆▆▂··▂···▃ beats 15/15	easy · avg 84%	▇ 0.6%	maxed out

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #115 of 393 · 11% of its score · 5 tests

Beats LongCat 2.0 (1.6T-A48B), Step 3.5 Flash (196B-A11B), GPT-5 mini here, though they rank higher overall. Trails LFM2.5 2.6B, Nemotron-Cascade-8B, Gemini 2.5 Flash here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena-Hard v2.0 hard prompts (style control, Gemini-2.5 judge)	79.1% ✎	▂·▃▅▅·▃█▆▂▃▂ beats 21/22	very hard · avg 12%	▇▇ 4.2%	sharp · at its level
LMArena (text)	1390.8 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 234/379	avg 1268	▇ 3.0%	blunt
Arena-Hard v2 aggregate	59.3% ✎	▁▁··▄▃▁▃█▁▁▇ beats 9/22	hard · avg 27%	▇ 1.9%	sharp · at its level
Arena-Hard v2.0 creative writing (GPT-4.1 + Gemini-2.5 judges)	58.7% ✎	▆▂▂▆▂▂▃█▂▃·▂ beats 14/22	hard · avg 19%	▇ 0.8%	sharp · at its level · noisy
IFEval (aggregation unspecified)	92.4% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 127/141	easy · avg 77%	▇ 0.7%	typical

math▁▂▃▅▆▅▄█▄▁▁▁ #80 of 398 · 23% of its score · 8 tests

Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.5 397B-A17B here, though they rank higher overall. Trails Granite 4.2 30B, INTELLECT-3, EXAONE 4.5 33B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
FrontierMath tiers 1–3 v2	36.1% ✎	▂··▁▁▃▄▄█▅▃▃ beats 26/82	very hard · avg 1.2%	▇▇▇▇ 11%	sharp
HMMT 2025 (edition unspecified)	66.7% ✎	▃█▇▅▇▅▁·▁··▁ beats 18/32	hard · avg 27%	▇ 2.4%	sharp
FrontierMath tier 4 v2	4.9% ✎	▁▂▃▂█▅▂▂▁▁▁▁ beats 9/63	very hard · avg 0.1%	▇ 2.4%	sharp · too hard for it
OTIS mock AIME 2024–2025	81.7% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 117/193	very hard · avg 8.5%	▇ 2.3%	sharp · at its level · noisy
MATH level 5	97.8% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 90/92	hard · avg 32%	▇ 2.2%	sharp · maxed out
MATH-500 (simple-evals) lab	98.2% ✎	▃··▅▅▃···▃·█ beats 9/9	medium · avg 61%	▇ 1.5%	sharp · maxed out
PolyMATH	48.7% ✎	▁···▃▁·▇█▄▄▄ beats 10/23	hard · avg 28%	▇ 0.8%	at its level
MGSM (simple-evals) lab	93.5% ✎	▁····▁▅▁█▁▃▃ beats 16/16	easy · avg 82%	▇ 0.7%	blunt

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #73 of 147 * · 2.1% of its score · 1 test

Beats DeepSeek V4 Flash Vision (Exp), GPT-5.4 nano, GPT-5 mini here, though they rank higher overall. Trails GPT-4.1, Gemma 4 12B, Qwen3.5 9B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena vision (style controlled)	1200.8 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 72/145	avg 1068	▇ 2.1%	blunt

web search▂▁▂▆▄█▃▃▁▁·▃ #2 of 80 * · 2.6% of its score · 1 test

Beats GPT-6 Astra, Claude Opus 5.5, Claude Fable 5.1 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
Finance Agent Benchmark v1 (537 questions, class-balanced accuracy)	37.3% ✎	█·▃▃·█▅▅▅▅▃▅ beats 16/18	very hard · avg 5.8%	▇ 2.6%	sharp · at its level

other tests25 not in a task · 34% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond (simple-evals) lab	81.3% ✎	█▃▃█·▃▃▅▃▃▃▅ beats 15/16	hard · avg 40%	▇▇ 6.1%	sharp · at its level
GPQA Diamond	79.6% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 126/208	hard · avg 40%	▇▇ 4.7%	sharp · at its level
KCL-Essay vanilla	31.3% ✎	▁···▁▃▂█▂▃▃▂ beats 9/23	hard · avg 18%	▇ 2.3%	at its level
EQ-Bench 3 (emotional intelligence, judge Elo)	1224.1 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 33/62	avg 1044	▇ 1.8%	typical
INCLUDE	80.8% ✎	▂····▂▂▄▄▄█▄ beats 30/36	medium · avg 59%	▇ 1.6%	typical
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	70.6% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 56/66	hard · avg 31%	▇ 1.4%	at its level
MMLU-Pro	81.9% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 141/188	medium · avg 59%	▇ 1.4%	typical
MMLU (simple-evals, zero-shot CoT) lab	90.3% ✎	▂▂·█▄▄▂·█▂·▂ beats 12/16	easy · avg 80%	▇ 1.4%	typical

17 more tests

test	result	vs the others in its run	difficulty	counts	why
MMLU-Redux	92.8% ✎	▁▁▃▁▂▃▄▅▇▅█▄ beats 50/63	easy · avg 81%	▇ 1.4%	typical
KCL-MCQA vanilla	37.7% ✎	▂···█▂▆▆██▆▂ beats 10/23	very hard · avg 30%	▇ 1.2%	at its level
OJBench	33.3% ✎	▁···▁▃·▅▄▅██ beats 19/26	very hard · avg 11%	▇ 1.2%	at its level
Chess Puzzles (Epoch)	26.0% ✎	█▁▁▁▁▂▁▄▂▂▁▁ beats 98/135	very hard · avg 5.9%	▇ 1.1%	sharp · noisy
HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge)	75.5% ✎	▁▁▁▁·▂·█▆█▇▁ beats 28/31	medium · avg 48%	▇ 1.1%	typical
Mystery Game Puzzles (Epoch)	5.0% ✎	█··▂▃▇▄█▂▃▁▁ beats 14/74	very hard · avg 9.4%	▇ 1.0%	sharp · too hard for it · noisy
SimpleQA Verified	18.8% ✎	▁▁▃▃▃▃▆▆█▃▂▅ beats 10/78	very hard · avg 8.0%	▇ 1.0%	at its level
MMLU-ProX	79.0% ✎	▁·▁▁▁▂▁▃▂█▅▄ beats 20/35	medium · avg 59%	▇ 0.9%	typical
Multi-IF (aggregation unspecified)	78.0% ✎	▁▁▁▁▁▁▃▄█▄▁▂ beats 28/41	medium · avg 60%	▇ 0.8%	blunt
Creative Writing v3	78.8% ✎	▂····▂▆··▆█▆ beats 3/14	medium · avg 65%	▇ 0.8%	typical
SuperGPQA	56.4% ✎	▂▁▂▂▅▆▃▇▇▇█▃ beats 29/46	medium · avg 37%	▇ 0.8%	at its level · small run
SimpleQA lab	24.0% ✎	▄▄▄▆█▆███·▂█ beats 22/32	very hard · avg 9.1%	▇ 0.7%	at its level · noisy
WritingBench	78.4% ✎	▁·▁▃▁··▅█▂▂▁ beats 2/20	easy · avg 66%	▇ 0.7%	typical
SimpleQA (simple-evals) lab	19.3% ✎	▄·▁▁▃▁··█▁·▁ beats 5/14	hard · avg 18%	▇ 0.3%	at its level · noisy
DROP F1, 3-shot (simple-evals) lab	78.1% ✎	▂▃▄▆▆█·▂···▄ beats 1/16	easy · avg 78%	▇ 0.2%	blunt
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	95.4% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 72/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	81.4% ✎	▁▃▁▂▅█▂▃▄▃▁▁ beats 7/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · $1.93/M · $1.1 in / $4.4 out per 1M tokens · 200k context · platform.openai.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

$ how the score was worked out · Epoch AI — internal evaluatio… 25%, Qwen/Qwen3-235B-A22B-Thinking… 23%, OpenAI simple-evals README be… 11% and 15 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Epoch AI — internal evaluations	8	▇▇▇▇▇▇▇▇ 25%	CC BY 4.0
Qwen/Qwen3-235B-A22B-Thinking-2507 model card	17	▇▇▇▇▇▇▇ 23%	apache-2.0
OpenAI simple-evals README benchmark results (OpenAI's own models) · lab-reported	7	▇▇▇ 11%	MIT
Aider polyglot benchmark runs	1	▇▇ 8%	Apache-2.0
LMArena leaderboard dataset	2	▇▇ 5%	CC BY 4.0
Arena-Hard-Auto v2.0-Preview leaderboard (repository README)	2	▇▇ 5%	Apache-2.0
τ²/τ³-bench leaderboard: Sierra's own evaluation runs	3	▇ 5%	MIT
Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities	2	▇ 3%	CC BY 4.0
Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)	1	▇ 3%	CC-BY-4.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	2	▇ 2%	Apache-2.0
LiveCodeBench official website results (2025-08-01)	1	▇ 2%	CC-BY-SA-4.0
EQ-Bench 3 canonical leaderboard results	1	▇ 2%	MIT
Berkeley Function Calling Leaderboard result archive	1	▇ 1%	Apache-2.0
OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · lab-reported	1	▇ 1%	Factual vendor-reported score (not licensed content)
WebApp1K models leaderboard (onekq-ai)	1	▇ 1%	MIT
OpenAI — GPT-5 System Card (arXiv:2601.03267v2) · lab-reported	1	▇ 1%	CC BY 4.0
Galileo Agent Leaderboard (v1 and v2 results)	1	▇ 0%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	2	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 66.3