whatmodel.sh / GPT-4o mini (2024-07-18)

GPT-4o mini (2024-07-18)

+ contribute

GPT-4o mini (2024-07-18) is a closed-weights language model from OpenAI, released July 2024. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-07-18). Dated API snapshot gpt-4o-mini-2024-07-18 (the only gpt-4o-mini snapshot).

53.1±0.3#295 of 719 LLMsfrom 107 benchmark results across 38 sources

$ how it compares

The LLMs ranked either side of GPT-4o mini (2024-07-18), out of 828 with a score, then more from OpenAI.

#	model	score	where	RAM	released
351	Qwen2-VL 72B Instruct Alibaba	53±1	local	~34GB	2024-09
352	Gemini 1.5 Flash Experimental (0827) Google	53±1	comparison only		2024-08
353	GPT-4o mini (2024-07-18) OpenAI	53±0	api $0.26/M		2024-07
354	Gemini 1.5 Pro (001) Google	53±1	comparison only		2024-05
355	Llama 3.1 Tulu 3 70B DPO Ai2	53±1	local	~22GB	2024-11
	more from OpenAI
1	GPT-6 Astra OpenAI	87±1	api $20.00/M		2026-09
3	GPT-6.1 Sol OpenAI	85±1	api $4.00/M		2026-09
7	GPT-6 Sol OpenAI	83±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #207 of 300 · 3.0% of its score · 9 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 (Jan '25) here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Mellum2 12B-A2.5B Instruct here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BrowserGym WebArena (AgentLab GenericAgent)	17.4% ✎	█··▄·▄█·▄··▄ beats 0/7	hard · avg 18%	▇ 0.6%	typical
BrowserGym WorkArena-L1 (AgentLab GenericAgent)	27.0% ✎	█···█▄▄·▄█·█ beats 0/10	hard · avg 29%	▇ 0.6%	at its level
BrowserGym MiniWoB (AgentLab GenericAgent)	56.6% ✎	▅····▅▅▃▃·▃█ beats 0/11	medium · avg 55%	▇ 0.4%	blunt · at its level
Finance Agent Benchmark v1 (537 questions, class-balanced accuracy)	10.8% ✎	█·▃▃·█▅▅▅▅▃▅ beats 6/18	very hard · avg 5.8%	▇ 0.4%	sharp
BFCL V4 (2025-11-03 archive, models absent from the December archive)	42.1% ✎	▃··▅▃▃·▅·▃·█ beats 4/10	hard · avg 34%	▇ 0.2%	at its level
BrowserGym WorkArena-L2 (AgentLab GenericAgent)	1.3% ✎	▃▅▃·▅▃▃··█·▃ beats 0/11	very hard · avg 3.9%	▇ 0.2%	sharp · too hard for it · noisy
BFCL v3	64.0% ✎	▁▁▁▁▁▃▅█▅▄▁▁ beats 93/114	medium · avg 52%	▇ 0.2%	blunt · at its level
Galileo Agent Leaderboard v1 (tool selection quality)	83.2% ✎	▃▃▃▃▆▅▆▅█▂▅▂ beats 13/31	easy · avg 79%	▇ 0.2%	blunt

1 more test

test	result	vs the others in its run	difficulty	counts	why
KlinikeBench strict pass@1 (333 interactive clinical encounters, Terminus-2)	0.6% ✎	▃▁·▁▁▁▁▅▃█▅█ beats 0/27	very hard · avg 2.4%	▇ 0.1%	too hard for it

coding▁▁▁▃▆██▇▆▄▃▁ #218 of 458 · 22% of its score · 10 tests

Beats Qwen3.5 4B, Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B) here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 3B Reasoning 2512, DeepSeek-Coder-V2 Instruct (0724) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
HumanEval+ +1	83.5% ✎	▁·▁▁▁▃▄▄▄█▃▃ beats 95/99	easy · avg 74%	▇▇ 6.0%	sharp · several runs
HumanEval	88.4% ✎	▁·▁▁▃▄▆▅█▃▁▁ beats 94/99	easy · avg 79%	▇▇ 5.4%	sharp
CRUXEval (ZeroEval, 800 problems)	75.9% ✎	▁▂▆▃▆█▆▃▂·▁▁ beats 39/47	medium · avg 61%	▇ 2.7%	sharp · at its level
Aider code editing (133 Exercism Python exercises, two attempts)	55.6% ✎	▁·▁▂▄▁▆▄█▆▂▂ beats 23/62	medium · avg 54%	▇ 2.0%	at its level
Aider polyglot (225 exercises, two attempts)	3.6% ✎	▃·▂▃▅▂▃█▆▄▂▂ beats 0/49	very hard · avg 8.5%	▇ 1.6%	sharp
MBPP+	72.2% ✎	▁···▁▁▂▅▇█▆▂ beats 67/75	easy · avg 68%	▇ 1.1%	at its level
HumanEval (Vector Institute, Inspect Evals)	86.0% ✎	▄·▄·█·█·█·▄▄ beats 2/9	easy · avg 79%	▇ 1.1%	typical
LiveCodeBench official (Aug 1, 2024–May 1, 2025)	27.5% ✎	▃▃▅▃··▃▅█▅▅▃ beats 1/15	hard · avg 25%	▇ 0.8%	at its level

2 more tests

test	result	vs the others in its run	difficulty	counts	why
HumanEval (simple-evals) lab +1	87.2% ✎	▆█▆▆▂··▂···▂ beats 3/15	easy · avg 84%	▇ 0.7%	several runs
WebApp1K-React pass@1 (onekq-ai)	82.7% ✎	▃▁▁█▁▄▅▃▄▇▁▄ beats 21/31	medium · avg 44%	▇ 0.5%	sharp · at its level · noisy

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #203 of 393 · 11% of its score · 7 tests

Beats Granite 4.2 8B, Qwen3-VL 30B-A3B Thinking, Mistral Small 4 (119B-A6.5B) here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, Gemma 3n E4B, LFM2.5 1.2B Thinking here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
AlpacaEval 2 length-controlled +1	50.7% ✎	▁▂▂▃▆▆█▆▄▄▁▁ beats 107/122	medium · avg 39%	▇▇ 5.6%	at its level · several runs
Arena-Hard (v0.1, GPT-4-1106 judge)	74.9% ✎	▂▁▁▅█▅▄▃▃▅▃▁ beats 47/60	medium · avg 52%	▇ 2.8%	sharp · at its level
LMArena (text)	1317.6 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 148/379	avg 1268	▇ 1.1%	blunt
Arabic IFEval prompt-level (Inception)	68.1% ✎	▁·▁▁▂▁▄▄▆▄█▄ beats 37/38	medium · avg 49%	▇ 0.9%	sharp · at its level
Arena-Hard (version/judge unspecified)	76.2% ✎	▃▃▂▅▃▆▅█▅▅▂▃ beats 24/30	easy · avg 66%	▇ 0.5%	at its level
IFEval prompt-level loose	83.5% ✎	▁▃▁▃▁▄▁▁█▃▃▁ beats 17/22	easy · avg 80%	▇ 0.2%	typical
IFEval (aggregation unspecified)	80.0% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 52/141	easy · avg 77%	▇ 0.1%	typical

math▁▂▃▅▆▅▄█▄▁▁▁ #240 of 398 · 13% of its score · 13 tests

Beats Command A (03-2025), GPT-4o (2024-11-20), GPT-4 Turbo (2024-04-09) here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 70B, OpenThinker 7B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH level 5	52.6% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 46/92	hard · avg 32%	▇ 2.7%	sharp · at its level
MATH (Vector Institute, Inspect Evals)	63.3% ✎	▄··██▄··▄·▄█ beats 2/9	medium · avg 53%	▇ 2.1%	sharp · at its level
MATH Level 5 (ZeroEval, 721 problems)	52.1% ✎	▁▁▁▆█▂▇▇▄··▂ beats 28/38	hard · avg 34%	▇ 1.6%	sharp · at its level
GSM8K +1	83.0% ✎	▂▃··▃▂▅█▆▃▂▃ beats 9/22	very easy · avg 91%	▇ 1.2%	sharp · maxed out · several runs
MGSM	81.7% ✎	▅▃▃█·▅···▃·▃ beats 10/10	easy · avg 71%	▇ 1.0%	typical
GSM8K (ZeroEval, zero-shot)	94.2% ✎	▁▄·▁▄█▅▅▅███ beats 33/43	very easy · avg 89%	▇ 0.9%	sharp · maxed out
MATH	70.2% ✎	█▄██···▄▄▄·▄ beats 10/10	medium · avg 64%	▇ 0.7%	at its level
MGSM (simple-evals) lab +1	87.0% ✎	▁····▁▅▁█▁▃▁ beats 4/16	easy · avg 82%	▇ 0.7%	blunt · several runs

5 more tests

test	result	vs the others in its run	difficulty	counts	why
OTIS mock AIME 2024–2025	6.9% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 41/193	very hard · avg 8.5%	▇ 0.6%	sharp · noisy
MATH (simple-evals, original test set, pre-o1 rows) lab	70.2% ✎	█····▄··▄··█ beats 2/5	medium · avg 63%	▇ 0.5%	at its level
FrontierMath tiers 1–3 v2	0.7% ✎	▃··▁▁▃▄▄█▅▃▃ beats 2/82	very hard · avg 1.2%	▇ 0.5%	sharp · too hard for it
MATH (subset unspecified)	67.9% ✎	▃▃█▂▃▆·····▁ beats 15/22	medium · avg 61%	▇ 0.4%	at its level
MathVista (testmini)	38.8% ✎	▁··▃▄▄▇█▅▆█▄ beats 1/47	medium · avg 55%	▇ 0.2%	at its level · noisy

vision & documents▁▁▃▂▃▃▄▂▆▇█▄ #118 of 147 · 7.1% of its score · 12 tests

Beats DeepSeek V4 Flash Vision (Exp) here, though they rank higher overall. Trails Qwen3.5 2B, Qwen3-VL 4B Instruct, Gemma 4 E4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv descriptive questions +1	74.9% ✎	▄█▄▄▄·██·▄·█ beats 9/12	easy · avg 71%	▇ 2.2%	several runs
CharXiv (reasoning questions) +1	34.1% ✎	▂▂▂█▃▂·▂▂··▂ beats 9/13	medium · avg 36%	▇ 1.5%	at its level · several runs
Arena vision (style controlled)	1097.1 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 30/145	avg 1068	▇ 0.7%	blunt
MMMU val (VHELM, zero-shot exact match)	54.8% ✎	▄▄··▄▄█··▄▄█ beats 3/9	medium · avg 51%	▇ 0.7%	at its level
MathVista testmini (VHELM, zero-shot exact match)	47.5% ✎	▃·▅···▃▃▃·▃█ beats 3/9	medium · avg 43%	▇ 0.6%	at its level
CapArena-Auto detailed image captioning (Score_Avg, GPT-4o judge)	9.3300 ✎	▃▃▃·▃█·▅▃▃▃▃ beats 7/12	avg -0.671	▇ 0.5%	typical
MMMU (val)	52.1% ✎	▁····▁▁▄█▇▆▆ beats 13/52	medium · avg 52%	▇ 0.4%	at its level
MathVista (testmini)	38.8% ✎	▁··▃▄▄▇█▅▆█▄ beats 1/47	medium · avg 55%	▇ 0.2%	at its level · noisy

4 more tests

test	result	vs the others in its run	difficulty	counts	why
PhysBench physical-world understanding (test, Avg)	43.1% ✎	▅▃█··▅▃····▃ beats 6/9	hard · avg 43%	▇ 0.2%	blunt · at its level
TextVQA	70.9% ✎	▁▁▁▄▅▁▃·▃▃▃█ beats 8/24	easy · avg 76%	▇ 0.1%	blunt
AI2D test	75.2% ✎	▁▁▅▇▃▇·▄·█·▃ beats 8/28	easy · avg 83%	▇ 0.0%	blunt
ChartQA	54.5% ✎	▁··▁▁▃▁▁▂▅▅█ beats 3/44	easy · avg 80%	▇ 0.0%	blunt · noisy

web search▂▁▂▆▄█▃▃▁▁·▁ #10 of 80 * · 0.4% of its score · 1 test

Beats Claude Opus 5.5, Claude Fable 5.1, Claude Sonnet 5.5 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
Finance Agent Benchmark v1 (537 questions, class-balanced accuracy)	10.8% ✎	█·▃▃·█▅▅▅▅▃▅ beats 6/18	very hard · avg 5.8%	▇ 0.4%	sharp

other tests42 not in a task · 44% of its score

test	result	vs the others in its run	difficulty	counts	why
LiveBench 2024-07-26	44.3% ✎	▁▁▁▂▂▃▂█▆▃▅▃ beats 47/61	medium · avg 41%	▇▇ 4.8%	at its level
MMLU-Redux (ZeroEval, 2,778 questions)	81.5% ✎	▁·▁▁▄▄▄▅█▄▂▁ beats 32/53	easy · avg 77%	▇▇ 4.2%	typical
MMLU-Pro +2	63.1% ✎	▃▂▅█▃·█▃▅··▂ beats 15/23	medium · avg 57%	▇ 3.6%	at its level · several runs
GPQA Diamond (simple-evals) lab +1	40.2% ✎	█▃▃█·▃▃▅▃▃▃▅ beats 0/16	hard · avg 40%	▇ 3.0%	sharp · at its level · several runs
BIG-Bench Hard +1	65.9% ✎	▁▄·▁▁·▅█·▄▄▁ beats 8/22	easy · avg 75%	▇ 2.5%	sharp · at its level · several runs
MMLU +1	82.2% ✎	▁▂·▁·▁▁█·▃▃▅ beats 17/22	easy · avg 77%	▇ 2.5%	at its level · several runs
ZebraLogic (ZeroEval, 1,000 puzzles)	20.1% ✎	▁▁▃▆██▁▁·▁▁▁ beats 30/57	hard · avg 18%	▇ 2.1%	sharp · at its level
GPQA Diamond	37.7% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 33/208	hard · avg 40%	▇ 1.6%	sharp · at its level

34 more tests

test	result	vs the others in its run	difficulty	counts	why
CLIcK Korean (Daekeun Kim)	68.5% ✎	▆▄▄▄█▄▂·▄█·▄ beats 8/23	medium · avg 58%	▇ 1.5%	at its level
GPQA Diamond (Vector Institute, Inspect Evals)	38.4% ✎	▃··▃▃▃▃·▃·▃█ beats 1/9	hard · avg 41%	▇ 1.4%	sharp · at its level
HAE-RAE 1.0 (Daekeun Kim)	76.4% ✎	▄▄▄▂▂█▂▄▂█▂▄ beats 8/22	medium · avg 63%	▇ 1.3%	at its level
WildBench v2 WB-Score (task macro)	57.1369 WB-Score ✎	▁·▁▁·▃█▄▇▆▅▆ beats 50/52	avg 45.199 WB-Score	▇ 1.2%	sharp
MMLU (simple-evals, zero-shot CoT) lab +1	82.0% ✎	▂▃·█▄▄▂·█▂·▂ beats 1/16	easy · avg 80%	▇ 1.2%	several runs
ARC Challenge	93.5% ✎	▂▂▂█▂···▄··▃ beats 10/10	very easy · avg 91%	▇ 1.2%	sharp
MMLU-Pro (Vector Institute, Inspect Evals)	64.0% ✎	▃····▃▅·▃▅·█ beats 1/9	medium · avg 60%	▇ 1.0%	at its level
OpenBookQA	90.0% ✎	▂··▄█▂·▂···▄ beats 9/10	easy · avg 87%	▇ 0.9%	typical
Multilingual MMLU (Microsoft)	72.9% ✎	█▄▄██··█···▄ beats 10/10	easy · avg 65%	▇ 0.8%	at its level
WinoGrande	76.9% ✎	▃····▃▃█▃·▅▅ beats 10/10	medium · avg 74%	▇ 0.7%	at its level
GPQA (subset unspecified)	41.1% ✎	█··▄▂··▂▂··▄ beats 10/10	very hard · avg 33%	▇ 0.6%	sharp · noisy
EvalPerf correctness (118 tasks, 20240328)	85.5% ✎	▂·▂▂▂▆▅▆▆▆▃█ beats 23/29	easy · avg 76%	▇ 0.6%	typical
KCL-Essay vanilla	16.1% ✎	▁···▃▃▂█▂▃▃▂ beats 1/23	hard · avg 18%	▇ 0.6%	typical
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	50.7% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 45/66	hard · avg 31%	▇ 0.6%	at its level
HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge)	65.5% ✎	▁▁▁▁·▂·█▆█▇▁ beats 17/31	medium · avg 48%	▇ 0.6%	at its level
Polish EQ-Bench v2 (SpeakLeash)	71.2% ✎	▁··▁▁▂▂▆▅▆█▆ beats 55/70	medium · avg 64%	▇ 0.5%	at its level
KMMLU-HARD (Daekeun Kim, zero-shot)	24.6% ✎	█···▁▁▁▂▃·▆▂ beats 1/23	very hard · avg 29%	▇ 0.5%	sharp
MMMLU lab	70.5% ✎	▂▁▁▂▃▂▄▄▅██▃ beats 20/46	medium · avg 66%	▇ 0.5%	at its level · small run
Open Persian LLM Leaderboard v2 average (21 tasks)	68.9% ✎	▂▄▂▃█▄▃▆▃▅▂▂ beats 69/81	medium · avg 58%	▇ 0.5%	blunt · at its level
Social IQA	82.9% ✎	▃·▃▅▅█·▃···▃ beats 10/10	easy · avg 78%	▇ 0.4%	blunt · at its level
BoolQ	88.7% ✎	▃··▃··██▃·▃▃ beats 10/10	easy · avg 85%	▇ 0.4%	typical
Judgemark v2 (LLM-as-judge for creative writing)	44.0% ✎	▃▃··▃▅▅···█▃ beats 1/10	medium · avg 55%	▇ 0.4%	at its level
HellaSwag	88.7% ✎	▃▅█·▃▃▃···▃▃ beats 10/10	easy · avg 84%	▇ 0.4%	typical
TruthfulQA MC2 +1	64.8% ✎	▃▃▂·▃▃·▅█▂▅▂ beats 15/21	medium · avg 63%	▇ 0.4%	blunt · at its level · several runs
PopQA	39.0% ✎	▁▁▁▃▃▃▁█▁▃·▄ beats 18/21	hard · avg 32%	▇ 0.3%	blunt · at its level
KCL-MCQA vanilla	29.9% ✎	▂···█▃▆▆██▆▂ beats 5/23	very hard · avg 30%	▇ 0.3%	typical
Open PT-LLM Leaderboard average (9 Portuguese tasks)	76.8% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 236/276	medium · avg 73%	▇ 0.2%	at its level · noisy
SimpleQA Verified	8.3% ✎	▁▃▃▂▃▃▆▆█▃▂▅ beats 1/78	very hard · avg 8.0%	▇ 0.2%	typical
DROP F1, 3-shot (simple-evals) lab +1	79.7% ✎	▂▂▄▆▆█·▂···▄ beats 3/16	easy · avg 78%	▇ 0.2%	blunt · several runs
PIQA	88.7% ✎	▃·▅·▃█·▃▅··▃ beats 10/10	easy · avg 83%	▇ 0.1%	blunt · at its level
SimpleQA (simple-evals) lab	9.5% ✎	▄·▁▁▁▁··█▁·▁ beats 2/14	hard · avg 18%	▇ 0.1%	noisy
Chess Puzzles (Epoch)	0.0% ✎	█▁▁▁▁▂▁▄▂▂▁▁ beats 12/135	very hard · avg 5.9%	▇ 0.1%	sharp · too hard for it · noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	98.3% ✎	▁▁▂▃▄▄█▆▃▃▂▁ beats 128/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy
Mystery Game Puzzles (Epoch)	12.0% ✎	█··▃▃▇▄█▂▃▁▁ beats 16/74	very hard · avg 9.4%	▇ 0.0%	sharp · too hard for it · noisy

$ where to run it

Through an API

api · $0.26/M · $0.15 in / $0.6 out per 1M tokens · 128k context · developers.openai.com ↗ · price sourceUSD per 1M tokens, OpenAI API standard processing, model page 'gpt-4o-mini' whose default snapshot is gpt-4o-mini-2024-07-18: input $0.15, output $0.6; cached-input, Batch and other tiers excluded. Currency: USD. OpenAI official model page, observed 2026-09-28.

$ how the score was worked out · EvalPlus / EvalPerf published… 12%, WildEval ZeroEval published r… 12%, microsoft/Phi-4-mini-instruct… 12% and 35 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
EvalPlus / EvalPerf published evaluation runs	4	▇▇▇▇ 12%	Apache-2.0
WildEval ZeroEval published results	5	▇▇▇ 12%	Apache-2.0
microsoft/Phi-4-mini-instruct model card	16	▇▇▇ 12%	MIT
Vector Institute Evaluation Leaderboard (Inspect Evals runs)	4	▇▇ 6%	Apache-2.0
Epoch AI — internal evaluations	7	▇▇ 6%	CC BY 4.0
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)	1	▇ 5%	Apache-2.0
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets	1	▇ 5%	Apache-2.0 (stated in the LiveBench datasheet)
2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models	8	▇ 5%	CC BY 4.0
Microsoft Research — Phi-4 Technical Report	9	▇ 5%	CC BY 4.0
OpenAI simple-evals README benchmark results (OpenAI's own models) · lab-reported	7	▇ 4%	MIT
Daekeun Kim Korean LLM benchmark runs	3	▇ 3%	MIT
Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)	1	▇ 3%	Apache-2.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	3	▇ 2%	Apache-2.0
Aider code editing benchmark runs (old edit leaderboard)	1	▇ 2%	Apache-2.0
ServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)	4	▇ 2%	MIT
CharXiv leaderboard (princeton-nlp/CharXiv README)	2	▇ 2%	Apache-2.0
MMLU-Pro author-released evaluation runs	1	▇ 2%	Apache-2.0
LMArena leaderboard dataset	2	▇ 2%	CC BY 4.0
OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · lab-reported	2	▇ 2%	Factual vendor-reported score (not licensed content)
Aider polyglot benchmark runs	1	▇ 2%	Apache-2.0
VHELM: A Holistic Evaluation of Vision Language Models (Stanford CRFM), arXiv 2410.07112v2	2	▇ 1%	CC BY 4.0
WildBench v2 leaderboard data (Ai2)	1	▇ 1%	Apache-2.0
Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities	2	▇ 1%	CC BY 4.0
LiveCodeBench official website results (2025-08-01)	1	▇ 1%	CC-BY-SA-4.0
Microsoft Phi-3.5-vision-instruct model card: Benchmarks table	5	▇ 1%	MIT
SpeakLeash Polish EQ-Bench leaderboard	1	▇ 1%	Apache-2.0
WebApp1K models leaderboard (onekq-ai)	1	▇ 1%	MIT
CapArena: Benchmarking and Analyzing Detailed Image Captioning in the LLM Era (arXiv:2503.12329v1)	1	▇ 1%	CC-BY-4.0
OpenAI — OpenAI o1 System Card (arXiv:2412.16720v2) · lab-reported	1	▇ 0%	CC BY 4.0
Berkeley Function Calling Leaderboard result archive	2	▇ 0%	Apache-2.0
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)	1	▇ 0%	Apache-2.0
Judgemark v2 results (EQ-Bench)	1	▇ 0%	MIT
Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)	1	▇ 0%	CC-BY-4.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 0%	Apache-2.0
Galileo Agent Leaderboard (v1 and v2 results)	1	▇ 0%	Apache-2.0
PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)	1	▇ 0%	CC-BY-4.0
KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)	1	▇ 0%	CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 53.1