whatmodel.sh / Claude Sonnet 4.5

Claude Sonnet 4.5

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Claude Sonnet 4.5 is a closed-weights language model from Anthropic, released September 2025.

68.5±0.6#100 of 719 LLMsfrom 50 benchmark results across 24 sources

$ how it compares

The LLMs ranked either side of Claude Sonnet 4.5, out of 828 with a score, then more from Anthropic.

#modelscorewhereRAMreleased
108GPT-5.5 Instant OpenAI69±1comparison only2026-05
109GPT-5.4 nano OpenAI69±1api $0.46/M2026-03
110Claude Sonnet 4.5 Anthropic69±1api $6.00/M2025-09
111Qwen3.5 27B Alibaba69±0api $0.84/M +local~18GB2026-02
112GLM-5V-Turbo Z.ai69±2api $1.90/M2026-04
more from Anthropic
2Claude Opus 5.5 Anthropic87±1api $8.00/M2026-09
4Claude Fable 5.1 Anthropic85±1api $20.00/M2026-09
5Claude Sonnet 5.5 Anthropic84±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #89 of 300 · 26% of its score · 14 tests

Beats Grok 4.20 (0309 reasoning), GPT-5.1, Grok 4.3 here, though they rank higher overall. Trails Hy3 preview (295B-A21B), Qwen3.6 35B-A3B, MiniMax-M2.7 here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
SWE-bench Verified +171.3% ✎▁··▁·▁▂▁▄█▄▁ beats 9/29hard · avg 19%▇▇▇ 8.5%sharp · at its level · several runs
Terminal-Bench 2.0 with Terminus 2 agent (benchmark authors' runs)42.8% ✎▃··▃·▃▃██▅▃▅ beats 12/14very hard · avg 5.8%▇ 2.9%sharp · at its level
ContractBench success rate (33 observation-contract tool tasks)69.7% ✎▆··▆▄▂▄▂▆▆█▄ beats 19/23very hard · avg 14%▇ 2.8%sharp · at its level
τ²-bench Retail +169.8% ✎█▄··▄··▄·▄·▄ beats 4/6medium · avg 37%▇ 1.7%at its level
τ³-bench banking_knowledge (v1.0.1 grading)25.3% ✎▃▃▂▃·▆▃▂█▅▅▂ beats 9/25very hard · avg 4.1%▇ 1.7%typical
τ³-bench airline (tau2-bench ≥ v1.0.1)72.0% ✎▃······▃·▃█▃ beats 0/6medium · avg 49%▇ 1.6%typical
BFCL V4 (December 2025 archive)73.2% ✎▁▁▁▂██▅▅▆▂▂▃ beats 76/77hard · avg 32%▇ 1.4%at its level
τ²-bench Airline +158.0% ✎▄▄······▄█▄▄ beats 3/6hard · avg 29%▇ 1.4%at its level
6 more tests
testresultvs the others in its rundifficultycountswhy
τ³-bench retail (tau2-bench ≥ v1.0.1)72.4% ✎▄▄·█··▄·▄··▄ beats 0/6medium · avg 46%▇ 1.4%at its level
τ³-bench telecom (tau2-bench ≥ v1.0.1)84.9% ✎█·▄█▄······▄ beats 0/6medium · avg 62%▇ 1.3%typical
KlinikeBench strict pass@1 (333 interactive clinical encounters, Terminus-2)9.9% ✎▁▁·▁▁▁▁▅▃█▅█ beats 10/27very hard · avg 2.4%▇ 0.6%typical
τ²-Bench-Verified (GPT-5.1 user)73.3% ✎▂··▄···▂▄▄█▄ beats 5/13medium · avg 45%▇ 0.6%at its level
T1-Bench multi-domain conversational agent pass^333.7% ✎▂▂··▂·▂·▂▂█▄ beats 6/11hard · avg 25%▇ 0.0%blunt · at its level · noisy
τ²-bench Telecom lab98.0% ✎▁·▅█▆▅▂▃▃▂▃▁ beats 82/89very hard · avg 9.6%—not counted yet: no other model in the same run to compare it with
coding▁▁▁▃▆██▇▆▄▃▁ #87 of 458 · 23% of its score · 5 tests

Beats Grok 4.20 (0309 reasoning), GPT-5.1, Grok 4.3 here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, GLM-5V-Turbo here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
SWE-bench Verified +171.3% ✎▁··▁·▁▂▁▄█▄▁ beats 9/29hard · avg 19%▇▇▇ 8.5%sharp · at its level · several runs
Aider polyglot (225 exercises, two attempts)78.0% ✎█▄····▄··█·▄ beats 6/6very hard · avg 7.9%▇▇▇ 6.7%sharp · at its level
Arena WebDev (Code Arena)1393.0 ✎▁▂▄▃▅█▄▇▆▄▁▁ beats 41/119avg 981▇▇ 5.3%sharp
VisEditBench strict pass rate (1,395 visualization-code editing tasks)62.7% ✎▄··█·▄··▄█·▄ beats 5/7hard · avg 34%▇ 1.5%at its level
WebApp1K-Duo-React pass@1 (onekq-ai)78.4% ✎▁··▁·▁▁▁▆█▇█ beats 44/45hard · avg 32%▇ 1.2%at its level
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #45 of 393 * · 3.0% of its score · 1 test

Beats MiMo-V2.6-Flash (309B-A15B), GPT-6 Luna, GPT-5.6 Luna here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1456.6 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 327/379avg 1268▇ 3.0%blunt
math▁▂▃▅▆▅▄█▄▁▁▁ #107 of 398 · 22% of its score · 6 tests

Beats Claude Sonnet 4.6, MiniMax-M3, Nemotron 3 Ultra 550B-A55B here, though they rank higher overall. Trails Gemini 2.5 Flash, Qwen3-VL 32B Thinking, gpt-oss-120b here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
FrontierMath tiers 1–3 v223.9% ✎▂··▁▁▃▄▄█▅▃▃ beats 15/82very hard · avg 1.2%▇▇▇▇▇ 13%sharp · at its level
FrontierMath tier 4 v22.4% ✎▁▃▁▂█▅▂▂▁▁▁▁ beats 6/63very hard · avg 0.1%▇▇ 3.9%sharp · too hard for it
OTIS mock AIME 2024–202577.8% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 112/193very hard · avg 8.5%▇ 1.8%sharp · noisy
MATH level 597.7% ✎▂▂▅▆▄█▆▃▅▂▃▄ beats 88/92hard · avg 32%▇ 1.4%sharp · maxed out
MATH-Vision71.1% ✎▁▁·▁··▄▁▃▇▇█ beats 9/24medium · avg 47%▇ 1.0%typical
FormalProofBench (200 graduate-level Lean 4 problems, agentic)18.0% ✎▃▅·▃··▅▃█··▃ beats 8/10very hard · avg 3.0%▇ 0.5%noisy
vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #66 of 147 · 4.1% of its score · 6 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5, Inkling Small (266B-A12B) here, though they rank higher overall. Trails Gemma 4 26B-A4B, Claude Haiku 4.5, Qwen3.6 Flash here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
VisEditBench strict pass rate (1,395 visualization-code editing tasks)62.7% ✎▄··█·▄··▄█·▄ beats 5/7hard · avg 34%▇ 1.5%at its level
Arena document (style controlled)1449.1 ✎▁▁▃▃▄▃▇█▅▇▄▅ beats 10/37avg 1335▇ 1.0%blunt
MATH-Vision71.1% ✎▁▁·▁··▄▁▃▇▇█ beats 9/24medium · avg 47%▇ 1.0%typical
Molmo2-VideoCount accuracy (val)27.2% ✎▃··▃·▃····█▃ beats 0/6hard · avg 22%▇ 0.3%blunt · at its level
Ai2 Molmo2 human preference Elo (video/image QA, 2026)1008.0 ✎▄··▄··▄··▄█▄ beats 0/6avg 975▇ 0.2%blunt
Molmo2-CapTest dense video captioning F126.0% ✎█···█·████·█ beats 0/6hard · avg 32%▇ 0.1%blunt · at its level
web search▂▁▂▆▄█▃▃▁▁·▁ #45 of 80 * · 1.2% of its score · 1 test

Beats Gemini 3.1 Pro (Preview), Qwen3.7 Max, GPT-5.2 here, though they rank higher overall. Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), Llama 4 Scout (109B-A17B) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Arena search systems (style controlled)1175.2 ✎▁·▁▁▂▂▃█▅▂▁▁ beats 9/30avg 1055▇ 1.2%blunt
other tests20 not in a task · 32% of its score
testresultvs the others in its rundifficultycountswhy
Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations)0.6258 ✎▁▄▃▃▇▄█▃▅▁▃▁ beats 29/30avg 0.423▇▇ 4.5%sharp
GPQA Diamond82.3% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 134/208hard · avg 40%▇▇ 4.1%sharp
EBR-bench (Epoch)2.4% ✎▄▂·▆█▂▆▄▆▆·▄ beats 0/23very hard · avg 0.3%▇ 2.5%sharp · too hard for it
ARC-AGI-1 public evaluation set (ARC Prize model_baseline, pass@2)73.8% ✎▁·▇▃▁·█▇▁·▇▃ beats 17/27very hard · avg 4.1%▇ 2.4%sharp · at its level · noisy
KCL-Essay vanilla47.6% ✎▁···▁▃▂█▂▃▃▂ beats 19/23hard · avg 18%▇ 2.3%at its level
Medmarks-V weighted mean win rate (61 models, 71 configurations)62.6% ✎▂▃▅▂▆▃█▅█▅▃▆ beats 33/34medium · avg 42%▇ 2.1%blunt · at its level
ARC-AGI-2 public evaluation set (ARC Prize model_baseline, pass@2)14.7% ✎█··▂▅▁▃▁▃·▁▃ beats 15/24very hard · avg 0.3%▇ 2.0%sharp · noisy
EQ-Bench 3 (emotional intelligence, judge Elo)1293.1 ✎▁·▂▁▄▅▅▃█▄▁▁ beats 44/62avg 1044▇ 1.8%typical
12 more tests
testresultvs the others in its rundifficultycountswhy
Mystery Game Puzzles (Epoch)17.0% ✎█··▂▃▇▄█▂▃▁▁ beats 26/74very hard · avg 9.4%▇ 1.5%sharp · too hard for it · noisy
Chess Puzzles (Epoch)12.0% ✎█▁▁▁▁▃▁▄▂▂▁▁ beats 60/135very hard · avg 5.9%▇ 1.4%sharp · noisy
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)78.2% ✎▁▁·▁▂▂▄█▄▅▄▄ beats 61/66hard · avg 31%▇ 1.3%at its level
RPC-Bench overall, text input (Z.ai)20.1% ✎▄··▄▄·▄▄█·▄▄ beats 0/8very hard · avg 11%▇ 1.3%at its level
KCL-MCQA vanilla54.9% ✎▂···█▂▆▆██▆▂ beats 21/23very hard · avg 30%▇ 1.2%at its level
LongSocialBench (full-context, 4-option MCQ over discussion threads)51.4% ✎▆▂·▄█·▂▂·▄▄▂ beats 12/16very hard · avg 36%▇ 1.1%blunt · at its level
SimpleQA Verified30.7% ✎▁▁▃▂▃▃▆▆█▃▂▅ beats 20/78very hard · avg 8.0%▇ 1.1%at its level
Multi-IF (aggregation unspecified)84.3% ✎▄·····▄█▄▄·▄ beats 4/6easy · avg 66%▇ 0.7%blunt
SOOHAK Challenge (340 problems, Avg@3)5.7% ✎█·█████··█·█ beats 0/7very hard · avg 4.1%▇ 0.5%typical
SOOHAK-Mini (702 problems, Avg@3)40.9% ✎▄··▄·▄·▄▄█·▄ beats 0/7medium · avg 36%▇ 0.4%blunt · at its level
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)94.5% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 55/140very easy · avg 94%▇ 0.1%blunt · maxed out · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)88.0% ✎▁▂▁▂▅█▂▃▄▃▁▁ beats 25/98very easy · avg 90%▇ 0.0%blunt

$ where to run it

Through an API

api · $6.00/M · $3 in / $15 out per 1M tokens · 200k context · docs.anthropic.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

$ how the score was worked out · Epoch AI — internal evaluatio… 37%, ServiceNow-AI/Apriel-1.6-15b-… 12%, LMArena leaderboard dataset 10% and 21 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Epoch AI — internal evaluations10▇▇▇▇▇▇▇▇▇▇▇ 37%CC BY 4.0
ServiceNow-AI/Apriel-1.6-15b-Thinker model card5▇▇▇▇ 12%MIT
LMArena leaderboard dataset4▇▇▇ 10%CC BY 4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models2▇▇ 7%CC BY 4.0
τ²/τ³-bench leaderboard: Sierra's own evaluation runs4▇▇ 6%MIT
ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets2▇ 4%MIT
Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities2▇ 4%CC BY 4.0
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials1▇ 3%CC BY 4.0
ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)1▇ 3%CC BY 4.0
EQ-Bench 3 canonical leaderboard results1▇ 2%MIT
VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback? (arXiv:2608.10408v1)1▇ 2%CC BY 4.0
Berkeley Function Calling Leaderboard result archive1▇ 1%Apache-2.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)1▇ 1%Apache-2.0
RPC-Bench leaderboard (Z.ai research paper comprehension)1▇ 1%Apache-2.0
WebApp1K models leaderboard (onekq-ai)1▇ 1%MIT
LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? (arXiv:2610.04118v1)1▇ 1%CC BY 4.0
Qwen/Qwen3.5-27B model card1▇ 1%apache-2.0
Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs — authors' own model runs2▇ 1%CC BY-SA 4.0
Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)3▇ 1%CC BY 4.0
KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)1▇ 1%CC BY 4.0
Amazon AGI — τ²-Bench-Verified1▇ 1%MIT
FormalProofBench (Vals AI, arXiv:2603.26996v1)1▇ 1%CC-BY-4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)2▇ 0%Apache-2.0
T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)1▇ 0%CC BY 4.0
prior · pull toward the middle▇ 0%
weighted average= 68.5

not counted (3)

τ-bench leaderboard: Anthropic's own submissionsτ²-bench Retail [correct]86.2%the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
τ-bench leaderboard: Anthropic's own submissionsτ²-bench Airline [correct]70.0%the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
τ-bench leaderboard: Anthropic's own submissionsτ²-bench Telecom [correct]98.0%fewer than three models have this test yet, or this source ran it for this model alone

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json