whatmodel.sh / Claude Opus 4
Claude Opus 4
↗ share
+ contribute
Claude Opus 4 is a closed-weights language model from Anthropic, released May 2025. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2025-05-22). Dated API snapshot claude-opus-4-20250514; distinct from Opus 4.1/4.5. Thinking budgets recorded per result. Anthropic model deprecations page: retired June 15, 2026.
$ how it compares
The LLMs ranked either side of Claude Opus 4, out of 828 with a score, then more from Anthropic.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 124 |  GPT-5 mini | 67±1 | api $0.69/M | ||
| 125 |  Gemini 2.5 Pro | 67±1 | api $3.44/M | ||
| 126 | Claude Opus 4 | 67±1 | comparison only | ||
| 127 |  K-EXAONE 2.0 (750B-A37B) | 67±1 | local | ||
| 128 | Claude Opus 4.1 | 67±1 | comparison only | ||
| more from Anthropic | |||||
| 2 | Claude Opus 5.5 | 87±1 | api $8.00/M | ||
| 4 | Claude Fable 5.1 | 85±1 | api $20.00/M | ||
| 5 | Claude Sonnet 5.5 | 84±1 | api $4.00/M | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #131 of 340 · 31% of its score · 4 tests
Beats o3, Gemini 2.5 Pro here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Large, Nova 2 Pro (Preview) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| SWE-bench Verified | 70.7% ✎ | ▁··▁·▁▂▁▄█▄▁ beats 8/29 | 16% |
| τ²-bench Telecom | 57.0% ✎ | ██··█·██··██ beats 5/6 | 7.1% |
| τ-bench Retail lab | 81.4% ✎ | ▁····▃·▂▇█▂▆ beats 27/28 | 4.3% |
| τ-bench Airline lab | 59.6% ✎ | ▁▁▁·▅▄▁█▇▁▁▇ beats 24/28 | 2.9% |
coding▁▁▁▃▅▇█▇▅▄▃▁ #111 of 524 · 42% of its score · 4 tests
Beats Grok 4.3, Muse Glimmer 30B, Qwen3.5 Plus here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Nemotron Cascade 2 30B-A3B, MiniMax-M2.1 here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) | 72.0% ✎ | ▂·▂▃▅▂▃█▆▄▂▂ beats 40/49 | 17% |
| SWE-bench Verified | 70.7% ✎ | ▁··▁·▁▂▁▄█▄▁ beats 8/29 | 16% |
| LiveCodeBench official (Aug 1, 2024–May 1, 2025) | 56.6% ✎ | ▃▃▅▃··▃▅█▅▅▃ beats 6/15 | 4.9% |
| WebApp1K-Duo-React pass@1 (onekq-ai) | 76.3% ✎ | ▁··▁·▁▁▁▆█▇█ beats 40/45 | 3.1% |
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #92 of 463 * · 6.9% of its score · 1 test
Beats LongCat 2.0 (1.6T-A48B), Inkling Small (266B-A12B), Muse Glimmer 30B here, though they rank higher overall. Trails Gemma 4 12B, Gemma 4 26B-A4B, Mistral Medium 3.5 (128B) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| LMArena (text) | 1426.1 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 278/379 | 6.9% |
math▁▂▃▅▇▅▅█▄▁▁▁ #193 of 462 * · 9.1% of its score · 2 tests
Trails OpenThinker3 7B, Olmo 3 7B Think, OpenThinker2 32B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| OTIS mock AIME 2024–2025 | 64.4% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 92/193 | 4.9% |
| MATH level 5 | 85.0% ✎ | ▂▂▅▆▄█▆▃▅▂▃▄ beats 72/92 | 4.3% |
vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #81 of 182 * · 4.8% of its score · 1 test
Beats DeepSeek V4 Flash Vision (Exp), GPT-5.4 nano, GPT-5 mini here, though they rank higher overall. Trails Gemma 4 12B, Qwen3.5 9B, Gemma 4 26B-A4B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena vision (style controlled) | 1207.4 ✎ | ▁▁▂▂▂▂▄▅▅▆██ beats 76/145 | 4.8% |
web search▂·▂▇▅█▃▄▁▂▁▁ #63 of 92 * · 2.8% of its score · 1 test
Beats Kimi K2.6, GPT-5 Pro, DeepSeek V4 Flash (Preview, Apr 2026) here, though they rank higher overall. Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), Llama 4 Scout (109B-A17B) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena search systems (style controlled) | 1150.6 ✎ | ▁·▁▁▃▂▂█▅▂▁▁ beats 5/30 | 2.8% |
other tests6 not in a task · 20% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond | 76.3% ✎ | ▁▁▁▁▂▄▆▄▅▇█▅ beats 117/208 | 10% |
| EQ-Bench 3 (emotional intelligence, judge Elo) | 1260.4 ✎ | ▁·▂▁▄▅▅▃█▄▁▁ beats 37/62 | 4.2% |
| AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge) | 81.0% ✎ | ▁▁·▁▂▂▄█▄▅▄▄ beats 63/66 | 3.2% |
| HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge) | 74.5% ✎ | ▁▁▁▁·▂·█▆█▇▁ beats 27/31 | 2.4% |
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 95.2% ✎ | ▁▁▂▃▄▄█▆▃▂▂▁ beats 65/140 | 0.2% |
| Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate) | 88.0% ✎ | ▁▂▁▂▅█▂▃▄▃▁▁ beats 26/98 | 0.0% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Epoch AI — internal evaluatio… 36%, Aider polyglot benchmark runs 17%, LMArena leaderboard dataset 14% and 7 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Epoch AI — internal evaluations | 4 | 36% |
 Aider polyglot benchmark runs | 1 | 17% |
 LMArena leaderboard dataset | 3 | 14% |
 τ-bench leaderboard: Anthropic's own submissions · lab-reported | 2 | 7% |
| Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table) | 1 | 7% |
| Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval) | 2 | 6% |
| LiveCodeBench official website results (2025-08-01) | 1 | 5% |
| EQ-Bench 3 canonical leaderboard results | 1 | 4% |
| WebApp1K models leaderboard (onekq-ai) | 1 | 3% |
| Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository) | 2 | 0% |
| prior · pull toward the middle | 0% | |
| weighted average | = 67.2 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json