whatmodel.sh / Claude 3.7 Sonnet

Claude 3.7 Sonnet

+ contribute

Claude 3.7 Sonnet is a closed-weights language model from Anthropic, released February 2025. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2025-02-24). Dated API snapshot claude-3-7-sonnet-20250219. Extended-thinking budgets (Epoch suffixes such as _64K) are API settings of the same checkpoint and are recorded per result. Anthropic model deprecations page: retired February 19, 2026.

65.3±0.7comparison only: no current run option verifiedfrom 35 benchmark results across 22 sources

$ how it compares

The LLMs ranked either side of Claude 3.7 Sonnet, out of 828 with a score, then more from Anthropic.

#	model	score	where	RAM	released
157	LongCat-Flash-Chat (560B-A27B) Meituan	66±1	local	~333GB	2025-08
158	o3-mini OpenAI	65±1	api $1.93/M		2025-01
159	Claude 3.7 Sonnet Anthropic	65±1	comparison only		2025-02
160	Claude Sonnet 4 Anthropic	65±1	comparison only		2025-05
161	MiMo-V2.6-Distill-Qwen-9B Xiaomi	65±2	local	~7.8GB	2026-09
	more from Anthropic
2	Claude Opus 5.5 Anthropic	87±1	api $8.00/M		2026-09
4	Claude Fable 5.1 Anthropic	85±1	api $20.00/M		2026-09
5	Claude Sonnet 5.5 Anthropic	84±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #199 of 340 · 28% of its score · 9 tests

Beats o1 (2024-12-17), o3-mini here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Mellum2 12B-A2.5B Instruct here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified	61.0% ✎	▁··▁·▁▃▁▄█▄▁ beats 4/29	hard · avg 19%	▇▇▇▇ 11%	sharp · at its level
Finance Agent Benchmark v1 (537 questions, class-balanced accuracy)	45.9% ✎	█·▃▃·█▅▅▅▅▃▅ beats 17/18	very hard · avg 5.8%	▇ 3.7%	sharp · at its level
BrowserGym WebArena (AgentLab GenericAgent)	44.6% ✎	█··▄·▄█·▄··▄ beats 7/7	hard · avg 18%	▇ 3.4%	at its level
τ²-bench Telecom	49.0% ✎	▁·▅█▆▅▂▃▃▂▃▁ beats 43/89	very hard · avg 13%	▇ 3.1%	sharp · at its level
τ²-bench Retail	72.1% ✎	▁····▂▁▃▆█▃▁ beats 32/44	medium · avg 41%	▇ 2.5%	at its level
τ²-bench Airline	64.2% ✎	▁▁·▂▁▃▂▁█▅▃▃ beats 37/42	hard · avg 29%	▇ 1.8%	at its level
Multi-SWE-bench, 7 languages (MagentLess scaffold)	5.1% ✎	▄···▄▄·▄▄·▄█ beats 4/7	very hard · avg 2.6%	▇ 1.1%	blunt · too hard for it
BFCL v3	58.6% ✎	▁▁▁▁▁▃▅█▅▄▁▁ beats 77/114	medium · avg 52%	▇ 0.9%	blunt · at its level

1 more test

test	result	vs the others in its run	difficulty	counts	why
Galileo Agent Leaderboard v1 (tool selection quality)	95.3% ✎	▃▃▃▃▆▅▆▅█▂▅▃ beats 31/31	easy · avg 79%	▇ 0.5%	blunt

coding▁▁▁▃▅▇█▇▅▄▃▁ #134 of 524 · 26% of its score · 4 tests

Beats Qwen3.5 Plus, Gemini 2.5 Pro, Kimi K2 Thinking here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Gemma 4 26B-A4B, Claude Haiku 4.5 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Aider polyglot (225 exercises, two attempts)	64.9% ✎	▂·▂▃▅▂▃█▆▄▂▂ beats 38/49	very hard · avg 8.5%	▇▇▇▇▇ 12%	sharp · at its level
SWE-bench Verified	61.0% ✎	▁··▁·▁▃▁▄█▄▁ beats 4/29	hard · avg 19%	▇▇▇▇ 11%	sharp · at its level
WebApp1K-Duo-React pass@1 (onekq-ai)	66.6% ✎	▁··▁·▁▁▁▆█▇█ beats 23/45	hard · avg 32%	▇ 2.1%	at its level
Multi-SWE-bench, 7 languages (MagentLess scaffold)	5.1% ✎	▄···▄▄·▄▄·▄█ beats 4/7	very hard · avg 2.6%	▇ 1.1%	blunt · too hard for it

chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #138 of 463 · 14% of its score · 4 tests

Beats LongCat 2.0 (1.6T-A48B), GPT-5 mini, MiMo-V2-Flash (309B-A15B) here, though they rank higher overall. Trails LFM2.5 2.6B, Nemotron-Cascade-8B, DeepSeek V3 0324 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena-Hard v2.0 hard prompts (style control, Gemini-2.5 judge)	59.8% ✎	▂·▃▅▅·▃█▆▂▂▂ beats 18/22	very hard · avg 12%	▇▇▇ 6.5%	sharp · at its level
LMArena (text)	1388.3 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 230/379	avg 1268	▇▇ 4.4%	blunt
IFEval prompt-level strict	87.3% ✎	▄▄··▄·▄··█·▄ beats 6/6	easy · avg 74%	▇ 1.5%	typical
Arena-Hard v2.0 creative writing (GPT-4.1 + Gemini-2.5 judges)	63.9% ✎	▆▂▂▆▂▂▃█▂▃·▂ beats 18/22	hard · avg 19%	▇ 1.2%	sharp · at its level · noisy

math▁▂▃▅▇▅▅█▄▁▁▁ #170 of 462 · 11% of its score · 5 tests

Beats MiMo-V2.5 (310B-A15B), MiniMax-M2.7, MiniMax-M2.1 here, though they rank higher overall. Trails Olmo 3 7B Think, Qwen3-VL 4B Thinking, Granite 4.2 3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
OTIS mock AIME 2024–2025	57.8% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 87/193	very hard · avg 8.5%	▇▇ 3.8%	sharp · at its level · noisy
MATH level 5	91.2% ✎	▂▂▅▆▄█▆▃▅▃▃▄ beats 79/92	hard · avg 32%	▇ 3.7%	sharp
AIME 2025	53.0% ✎	█··▄··█··▄·▄ beats 1/6	hard · avg 34%	▇ 1.7%	sharp · noisy
AIME 2024	55.3% ✎	▄···▄█·▄▄··▄ beats 0/6	medium · avg 43%	▇ 1.1%	sharp · noisy
THINK-Bench final-answer accuracy (reasoning models)	94.3% ✎	▄█·▄····▄▄·█ beats 6/7	easy · avg 76%	▇ 0.6%	typical

vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #88 of 182 · 6.0% of its score · 4 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5.4 nano, GPT-5 mini here, though they rank higher overall. Trails GPT-4.1, Gemma 4 12B, Qwen3.5 9B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena vision (style controlled)	1192.8 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 66/145	avg 1068	▇ 3.0%	blunt
CharXiv (reasoning questions)	70.9% ✎	▄·····▄▄·▄·█ beats 4/5	hard · avg 33%	▇ 1.5%	at its level
CharXiv descriptive questions	93.3% ✎	▄····▄···█▄▄ beats 5/5	easy · avg 68%	▇ 0.8%	noisy
MSEarth-Bench Earth-science figure MCQ (Overall accuracy)	58.0% ✎	█·██····██·█ beats 4/5	hard · avg 48%	▇ 0.8%	blunt · at its level

web search▂·▂▇▅█▃▄▁▂▁▃ #1 of 92 * · 3.7% of its score · 1 test

Beats GPT-6 Astra, Claude Opus 5.5, Claude Fable 5.1 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
Finance Agent Benchmark v1 (537 questions, class-balanced accuracy)	45.9% ✎	█·▃▃·█▅▅▅▅▃▅ beats 17/18	very hard · avg 5.8%	▇ 3.7%	sharp · at its level

other tests10 not in a task · 27% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond +1	79.7% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 127/208	hard · avg 40%	▇▇▇▇▇ 14%	sharp · at its level · several runs
KCL-Essay vanilla	43.6% ✎	▁···▁▃▂█▂▃▃▂ beats 17/23	hard · avg 18%	▇ 3.3%	at its level
EQ-Bench 3 (emotional intelligence, judge Elo)	1169.2 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 25/62	avg 1044	▇ 2.7%	typical
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	78.2% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 60/66	hard · avg 31%	▇ 2.2%	at its level
KCL-MCQA vanilla	50.4% ✎	▂···█▂▆▆██▆▂ beats 18/23	very hard · avg 30%	▇ 1.7%	at its level
HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge)	70.8% ✎	▁▁▁▁·▂·█▆█▇▁ beats 20/31	medium · avg 48%	▇ 1.7%	typical
DiagnosticIQ industrial maintenance MCQ (macro accuracy, 4 options)	70.6% ✎	▁··▃▁▁▁▅█▅▁▅ beats 21/24	medium · avg 52%	▇ 0.9%	blunt · at its level
Open PT-LLM Leaderboard average (9 Portuguese tasks)	84.5% ✎	▃▁▁▁▁▁▁▁▂█▅▃ beats 274/276	medium · avg 73%	▇ 0.5%	noisy

2 more tests

test	result	vs the others in its run	difficulty	counts	why
Omni-MATH	54.6% ✎	█·█·█·██··██ beats 0/6	medium · avg 64%	▇ 0.2%	blunt · noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	95.5% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 75/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Epoch AI — internal evaluatio… 26%, Aider polyglot benchmark runs 12%, Microsoft — Phi-4-reasoning T… 9% and 19 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Epoch AI — internal evaluations	4	▇▇▇▇▇▇▇▇ 26%	CC BY 4.0
Aider polyglot benchmark runs	1	▇▇▇▇ 12%	Apache-2.0
Microsoft — Phi-4-reasoning Technical Report	4	▇▇▇ 9%	CC BY 4.0
Arena-Hard-Auto v2.0-Preview leaderboard (repository README)	2	▇▇ 8%	Apache-2.0
LMArena leaderboard dataset	2	▇▇ 7%	CC BY 4.0
τ²/τ³-bench leaderboard: Sierra's own evaluation runs	3	▇▇ 7%	MIT
Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities	2	▇▇ 5%	CC BY 4.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	2	▇ 4%	Apache-2.0
Finance Agent Benchmark (Vals AI, arXiv:2508.00828v1)	1	▇ 4%	CC-BY-4.0
ServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)	1	▇ 3%	MIT
EQ-Bench 3 canonical leaderboard results	1	▇ 3%	MIT
ServiceNow-AI/Apriel-1.6-15b-Thinker model card	2	▇ 2%	MIT
WebApp1K models leaderboard (onekq-ai)	1	▇ 2%	MIT
OpenCompass CompassAcademic leaderboard (full version) Space: committed predictions file	1	▇ 1%	Apache-2.0
Multi-SWE-bench experiments (ByteDance Seed baseline runs)	1	▇ 1%	Apache-2.0
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)	1	▇ 1%	CC BY 4.0
Berkeley Function Calling Leaderboard result archive	1	▇ 1%	Apache-2.0
MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (arXiv:2505.20740v3)	1	▇ 1%	CC-BY-4.0
THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models (arXiv:2505.22113v1)	1	▇ 1%	CC-BY-4.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
Galileo Agent Leaderboard (v1 and v2 results)	1	▇ 0%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 65.3