Claude 3 Opus

+ contribute

Claude 3 Opus is a closed-weights language model from Anthropic, released February 2024. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-02-29). Dated API snapshot claude-3-opus-20240229. Anthropic model deprecations page: retired January 5, 2026.

52.5±0.4comparison only: no current run option verifiedfrom 37 benchmark results across 21 sources

$ how it compares

The LLMs ranked either side of Claude 3 Opus, out of 828 with a score, then more from Anthropic.

#	model	score	where	RAM	released
367	Ministral 3 8B Mistral	53±1	api $0.15/M +local	~6.5GB	2025-12
368	Qwen3-VL 4B Instruct Alibaba	53±1	local	~4.1GB	2025-10
369	Claude 3 Opus Anthropic	53±0	comparison only		2024-02
370	LFM2.5 2.6B Liquid AI	53±1	local	~3.2GB	2026-07
371	Bielik 11B v2.1 Instruct SpeakLeash	53±3	local	~8.5GB	2024-08
	more from Anthropic
2	Claude Opus 5.5 Anthropic	87±1	api $8.00/M		2026-09
4	Claude Fable 5.1 Anthropic	85±1	api $20.00/M		2026-09
5	Claude Sonnet 5.5 Anthropic	84±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #256 of 340 * · 0.4% of its score · 1 test

Beats o3-mini, Llama 3.1 Nemotron Ultra 253B v1, Nova 2 Lite here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, LFM2.5 1.2B Thinking here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BFCL v3	57.0% ✎	▁▁▁▁▁▃▅█▅▄▁▁ beats 68/114	medium · avg 52%	▇ 0.4%	blunt · at its level

coding▁▁▁▃▅▇█▇▅▄▃▁ #275 of 524 · 36% of its score · 7 tests

Beats Qwen3.5 4B, Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B) here, though they rank higher overall. Trails QwQ-32B-Preview, Gemma 2 27B IT, Ministral 3 3B Reasoning 2512 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
HumanEval +1	82.9% ✎	▁·▁▁▃▄▆▅█▃▁▁ beats 88/99	easy · avg 79%	▇▇▇▇▇▇ 14%	sharp · several runs
HumanEval+	77.4% ✎	▁·▁▁▁▃▄▄▄█▃▁ beats 88/99	easy · avg 74%	▇▇▇ 7.9%	sharp
CRUXEval (ZeroEval, 800 problems)	70.4% ✎	▁▂▆▃▆█▆▃▂·▁▁ beats 30/47	medium · avg 61%	▇▇ 4.3%	sharp · at its level
LiveCodeBench official (Jan 1–Nov 1, 2024)	31.9% ✎	▁▁·▁▂▄▆█▅▂·▂ beats 16/31	hard · avg 26%	▇▇ 4.1%	sharp · at its level
Aider code editing (133 Exercism Python exercises, two attempts)	68.4% ✎	▁·▁▂▄▁▆▄█▆▂▂ beats 45/62	medium · avg 54%	▇ 3.1%	at its level
MBPP+	73.3% ✎	▁···▁▁▂▅▇█▆▂ beats 68/75	easy · avg 68%	▇ 1.7%	at its level
SciCode main problems (official leaderboard)	1.5% ✎	▃···█··▂·▃·▂ beats 6/17	very hard · avg 1.1%	▇ 0.6%	sharp · too hard for it

chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #248 of 463 · 15% of its score · 4 tests

Beats Granite 4.2 8B, Mistral Small 4 (119B-A6.5B), Gemma 2 9B IT SimPO here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, Granite 4.0 H Micro 3B, Gemma 3n E4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
AlpacaEval 2 length-controlled	40.5% ✎	▁▂▂▃▆▆█▆▄▄▁▁ beats 98/122	medium · avg 39%	▇▇▇ 7.5%	at its level
Arena-Hard (v0.1, GPT-4-1106 judge)	60.4% ✎	▂▁▁▅█▅▄▃▃▅▃▁ beats 41/60	medium · avg 52%	▇▇ 4.4%	sharp · at its level
Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024)	1273.0 ✎	▂▁▂▁▃▇█▆▇▄▂▃ beats 93/95	avg 1201	▇ 1.7%	blunt
LMArena (text)	1322.1 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 156/379	avg 1268	▇ 1.7%	blunt

math▁▂▃▅▇▅▅█▄▁▁▁ #321 of 462 · 9.0% of its score · 4 tests

Beats Molmo 2 8B, DeepSeek-Coder-V2 Instruct (0724) here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Granite 8B Code Base (4K), Mixtral 8x7B v0.1 (base) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH level 5	37.5% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 29/92	hard · avg 32%	▇▇ 4.2%	sharp · at its level
MATH Level 5 (ZeroEval, 721 problems)	36.9% ✎	▁▁▁▆█▃▇▇▄··▂ beats 18/38	hard · avg 34%	▇ 2.5%	sharp · at its level
GSM8K (ZeroEval, zero-shot)	95.6% ✎	▁▄·▁▄█▅▅▅███ beats 38/43	very easy · avg 89%	▇ 1.5%	sharp
OTIS mock AIME 2024–2025	4.7% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 31/193	very hard · avg 8.5%	▇ 0.8%	sharp · noisy

vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #153 of 182 · 8.2% of its score · 9 tests

Trails Qwen3.5 2B, Qwen3-VL 2B Thinking, Molmo 72B (0924) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv descriptive questions	71.5% ✎	▄█▄▄▄·██·▄·█ beats 6/12	easy · avg 71%	▇ 1.8%	typical
Arena vision (style controlled)	1061.7 ✎	▁▁▂▂▂▃▄▅▅▆██ beats 23/145	avg 1068	▇ 1.2%	blunt
CharXiv (reasoning questions)	30.2% ✎	▂▂▃█▃▂·▂▂··▂ beats 2/13	medium · avg 36%	▇ 1.2%	at its level
MMMU val (VHELM, zero-shot exact match)	53.2% ✎	▄▄··▄▄█··▄▄█ beats 2/9	medium · avg 51%	▇ 1.1%	at its level
CharXiv descriptive questions (test split)	72.2% ✎	█··▄··█··▄·▄ beats 3/6	easy · avg 71%	▇ 1.0%	at its level
MathVista testmini (VHELM, zero-shot exact match)	40.5% ✎	▃·▅···▃▃▃·▃█ beats 2/9	medium · avg 43%	▇ 1.0%	at its level
CharXiv reasoning questions (test split)	28.8% ✎	▄··▄▄▄█····▄ beats 2/6	hard · avg 32%	▇ 0.6%	blunt · at its level
PhysBench physical-world understanding (test, Avg)	37.0% ✎	▅▃█··▅▃····▃ beats 1/9	hard · avg 43%	▇ 0.2%	blunt · at its level

1 more test

test	result	vs the others in its run	difficulty	counts	why
Ai2 Molmo human preference Elo (Sept 2024, 27 VLMs)	971.0 ✎	▄▄·▄·▄▄▄·▄·█ beats 0/8	avg 1037	▇ 0.2%	blunt

other tests11 not in a task · 31% of its score

test	result	vs the others in its run	difficulty	counts	why
LiveBench 2024-07-26	50.6% ✎	▁▁▁▂▂▃▂█▆▃▅▃ beats 53/61	medium · avg 41%	▇▇▇ 7.4%	at its level
MMLU-Redux (ZeroEval, 2,778 questions)	82.5% ✎	▁·▁▁▄▄▄▅█▄▂▁ beats 37/53	easy · avg 77%	▇▇▇ 6.7%	at its level
EvoEval (overall)	62.9% ✎	▂▂▃▄▄█▁▄▅▃▁▃ beats 40/42	medium · avg 55%	▇▇ 5.1%	at its level
ZebraLogic (ZeroEval, 1,000 puzzles)	27.0% ✎	▁▁▃▆██▁▁·▁▁▁ beats 42/57	hard · avg 18%	▇ 3.1%	sharp
GPQA Diamond	47.2% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 56/208	hard · avg 40%	▇ 2.4%	sharp · at its level
BigCodeBench (Instruct)	45.2% ✎	▃▂·▆▆██▆▅▃▃▃ beats 30/33	medium · avg 41%	▇ 2.2%	at its level
BigCodeBench (Complete)	56.8% ✎	▂▁▁▄▂▅▇█▆▂▂▃ beats 56/58	medium · avg 50%	▇ 2.0%	at its level
WildBench v2 WB-Score (task macro)	51.7140 WB-Score ✎	▁·▁▁·▃█▄▇▆▅▆ beats 41/52	avg 45.199 WB-Score	▇ 1.9%	sharp

3 more tests

test	result	vs the others in its run	difficulty	counts	why
SimpleQA Verified	12.6% ✎	▁▁▃▂▃▃▆▆█▃▂▅ beats 5/78	very hard · avg 8.0%	▇ 0.2%	typical
Chess Puzzles (Epoch)	5.0% ✎	█▁▁▁▁▂▁▄▂▂▁▁ beats 16/135	very hard · avg 5.9%	▇ 0.1%	sharp · too hard for it · noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	89.9% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 25/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · EvalPlus / EvalPerf published… 18%, WildEval ZeroEval published r… 18%, Top Leaderboard Ranking = Top… 10% and 18 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
EvalPlus / EvalPerf published evaluation runs	3	▇▇▇▇▇▇ 18%	Apache-2.0
WildEval ZeroEval published results	5	▇▇▇▇▇ 18%	Apache-2.0
Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM — authors' own runs	2	▇▇▇ 10%	CC BY 4.0
Epoch AI — internal evaluations	5	▇▇ 8%	CC BY 4.0
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)	1	▇▇ 7%	Apache-2.0
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets	1	▇▇ 7%	Apache-2.0 (stated in the LiveBench datasheet)
Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)	1	▇ 4%	Apache-2.0
BigCodeBench paper (Zhuo et al., ICLR 2025), arXiv 2406.15877v4	2	▇ 4%	CC BY 4.0
LiveCodeBench official website results (2024-12-26)	1	▇ 4%	CC-BY-SA-4.0
Aider code editing benchmark runs (old edit leaderboard)	1	▇ 3%	Apache-2.0
CharXiv leaderboard (princeton-nlp/CharXiv README)	2	▇ 3%	Apache-2.0
LMArena leaderboard dataset	2	▇ 3%	CC BY 4.0
VHELM: A Holistic Evaluation of Vision Language Models (Stanford CRFM), arXiv 2410.07112v2	2	▇ 2%	CC BY 4.0
WildBench v2 leaderboard data (Ai2)	1	▇ 2%	Apache-2.0
Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3)	1	▇ 2%	CC-BY-4.0
CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (arXiv:2406.18521v1)	2	▇ 2%	CC BY-SA 4.0
SciCode leaderboard (repository README)	1	▇ 1%	Apache-2.0
Berkeley Function Calling Leaderboard result archive	1	▇ 0%	Apache-2.0
PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)	1	▇ 0%	CC-BY-4.0
Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column	1	▇ 0%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 52.5