whatmodel.sh / GPT-4o (2024-11-20)

GPT-4o (2024-11-20)

+ contribute

GPT-4o (2024-11-20) is a closed-weights language model from OpenAI, released November 2024. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-11-20). Dated API snapshot gpt-4o-2024-11-20, listed among gpt-4o snapshots; not the default snapshot, so no price is attached.

57.2±0.7#233 of 719 LLMsfrom 31 benchmark results across 13 sources

$ how it compares

The LLMs ranked either side of GPT-4o (2024-11-20), out of 828 with a score, then more from OpenAI.

#	model	score	where	RAM	released
276	Granite 4.2 3B IBM	57±1	api $0.052/M +local	~3.7GB	2026-08
277	Command A (03-2025) Cohere	57±1	api $4.38/M +local	~37GB	2025-03
278	GPT-4o (2024-11-20) OpenAI	57±1	api $4.38/M		2024-11
279	Gemini 2.5 Flash-Lite Google	57±2	api $0.17/M		2025-07
280	Qwen2.5-Max (2025-01-25) Alibaba	57±1	comparison only		2025-01
	more from OpenAI
1	GPT-6 Astra OpenAI	87±1	api $20.00/M		2026-09
3	GPT-6.1 Sol OpenAI	85±1	api $4.00/M		2026-09
7	GPT-6 Sol OpenAI	83±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #162 of 300 · 40% of its score · 10 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 0528 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
τ²-bench Telecom +2	24.1% ✎	█··██··██··█ beats 1/5	very hard · avg 10%	▇▇▇▇▇▇ 14%	sharp · several runs
SWE-bench Verified	31.0% ✎	▃··▁·▁▂▁▄█▄▁ beats 0/29	hard · avg 19%	▇▇▇▇ 9.3%	sharp · at its level
τ-bench Retail +1	60.3% ✎	▁····▃·▂▇█▂▆ beats 10/28	medium · avg 40%	▇▇ 5.0%	at its level · small run · several runs
τ²-bench Retail +1	66.7% ✎	▁····▂▁▃▆█▃▁ beats 21/44	medium · avg 45%	▇ 3.1%	at its level · small run · several runs
τ-bench Airline +1	42.8% ✎	▁▁▁·▅▄▁█▇▁▁▇ beats 11/28	hard · avg 22%	▇ 2.6%	at its level · small run · several runs
τ²-bench Airline +1	42.0% ✎	▁▁·▂▃▃▂▁█▅▂▃ beats 6/42	hard · avg 26%	▇ 2.5%	at its level · small run · several runs
BFCL V4 (2025-11-03 archive, models absent from the December archive)	52.2% ✎	▃··▅▃▃·▅·▃·█ beats 5/10	hard · avg 34%	▇ 0.9%	at its level
BFCL v3	71.7% ✎	▁▁▁▁▁▃▅█▅▄▃▁ beats 110/114	medium · avg 52%	▇ 0.9%	blunt · at its level

2 more tests

test	result	vs the others in its run	difficulty	counts	why
Multi-SWE-bench, 7 languages (MagentLess scaffold)	3.8% ✎	▄···▄▄·▄▄·▄█ beats 2/7	very hard · avg 2.6%	▇ 0.7%	blunt · too hard for it
Galileo Agent Leaderboard v1 (tool selection quality)	90.0% ✎	▃▃▃▃▆▅▆▅█▂▅▂ beats 23/31	easy · avg 79%	▇ 0.6%	blunt

coding▁▁▁▃▆██▇▆▄▃▁ #209 of 458 · 27% of its score · 5 tests

Beats Qwen3.5 4B, Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B) here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 3B Reasoning 2512, DeepSeek-Coder-V2 Instruct (0724) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Aider polyglot (225 exercises, two attempts)	18.2% ✎	▂·▂▃▅▂▃█▆▄▂▂ beats 10/49	very hard · avg 8.5%	▇▇▇▇ 9.4%	sharp · at its level
SWE-bench Verified	31.0% ✎	▃··▁·▁▂▁▄█▄▁ beats 0/29	hard · avg 19%	▇▇▇▇ 9.3%	sharp · at its level
Aider code editing (133 Exercism Python exercises, two attempts)	71.4% ✎	▁·▁▂▄▁▆▄█▆▂▂ beats 50/62	medium · avg 54%	▇▇▇ 6.4%	at its level
HumanEval (simple-evals) lab	90.2% ✎	▆█▆▆▂··▂···▂ beats 8/15	easy · avg 84%	▇ 1.3%	typical
Multi-SWE-bench, 7 languages (MagentLess scaffold)	3.8% ✎	▄···▄▄·▄▄·▄█ beats 2/7	very hard · avg 2.6%	▇ 0.7%	blunt · too hard for it

math▁▂▃▅▆▅▄█▄▁▁▁ #248 of 398 · 14% of its score · 3 tests

Trails DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 70B, OpenThinker 7B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH level 5	49.8% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 41/92	hard · avg 32%	▇▇▇▇ 9.1%	sharp · at its level
OTIS mock AIME 2024–2025	6.3% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 34/193	very hard · avg 8.5%	▇ 3.5%	sharp · at its level · noisy
MGSM (simple-evals) lab	90.3% ✎	▁····▁▅▁█▁▃▁ beats 12/16	easy · avg 82%	▇ 1.3%	blunt

vision & documents▁▁▃▂▁▃▄▃▆▇█▄ #92 of 147 * · 6.2% of its score · 2 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5 nano, Magistral Small 1.2 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv descriptive questions lab	85.3% ✎	▄▄·····▄·▄█▄ beats 2/6	easy · avg 72%	▇ 3.4%	typical
CharXiv (reasoning questions) lab	52.7% ✎	▄··▃█··▂···▄ beats 2/9	hard · avg 34%	▇ 2.7%	at its level

other tests7 not in a task · 23% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond (simple-evals) lab	46.0% ✎	█▃▃█·▃▃▅▃▃▃▅ beats 3/16	hard · avg 40%	▇▇▇▇ 8.9%	sharp · at its level
GPQA Diamond	47.9% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 62/208	hard · avg 40%	▇▇▇ 7.1%	sharp · at its level
MMLU (simple-evals, zero-shot CoT) lab	85.7% ✎	▂▂·█▄▄▂·█▂·▂ beats 5/16	easy · avg 80%	▇ 2.7%	typical
HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge)	72.4% ✎	▁▁▁▁·▂·█▆█▇▁ beats 22/31	medium · avg 48%	▇ 2.0%	at its level
Judgemark v2 (LLM-as-judge for creative writing)	82.5% ✎	▃▃··▃▅▅···█▃ beats 10/10	medium · avg 55%	▇ 1.4%	at its level
DROP F1, 3-shot (simple-evals) lab	81.5% ✎	▂▂▄▆▆█·▂···▄ beats 8/16	easy · avg 78%	▇ 0.4%	blunt
SimpleQA (simple-evals) lab	38.8% ✎	▄·▁▁▁▁··█▁·▁ beats 7/14	hard · avg 18%	▇ 0.3%	at its level · noisy

$ where to run it

Through an API

api · $4.38/M · $2.5 in / $10 out per 1M tokens · 128k context · developers.openai.com ↗ · price sourceStandard base token rate from models.dev (revision 6b32416, observed 2026-10-07). Provider-specific tiers, long-context surcharges, caching and promotions may differ.

$ how the score was worked out · Epoch AI — internal evaluatio… 29%, OpenAI simple-evals README be… 15%, Qwen/Qwen3-235B-A22B-Instruct… 10% and 10 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Epoch AI — internal evaluations	4	▇▇▇▇▇▇▇▇▇ 29%	CC BY 4.0
OpenAI simple-evals README benchmark results (OpenAI's own models) · lab-reported	6	▇▇▇▇ 15%	MIT
Qwen/Qwen3-235B-A22B-Instruct-2507 model card	5	▇▇▇ 10%	apache-2.0
Aider polyglot benchmark runs	1	▇▇▇ 9%	Apache-2.0
Qwen/Qwen3-30B-A3B-Instruct-2507 model card	5	▇▇▇ 9%	apache-2.0
MUA-RL: Multi-turn User-interacting Agent Reinforcement Learning for agentic tool use (arXiv:2508.18669v1)	1	▇▇▇ 8%	CC-BY-4.0
Aider code editing benchmark runs (old edit leaderboard)	1	▇▇ 6%	Apache-2.0
OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · lab-reported	2	▇▇ 6%	Factual vendor-reported score (not licensed content)
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	1	▇ 2%	Apache-2.0
Berkeley Function Calling Leaderboard result archive	2	▇ 2%	Apache-2.0
Judgemark v2 results (EQ-Bench)	1	▇ 1%	MIT
Multi-SWE-bench experiments (ByteDance Seed baseline runs)	1	▇ 1%	Apache-2.0
Galileo Agent Leaderboard (v1 and v2 results)	1	▇ 1%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 57.2