whatmodel.sh / Claude Haiku 4.5

Claude Haiku 4.5

+ contribute

Claude Haiku 4.5 is a closed-weights language model from Anthropic, released October 2025.

64.2±0.8#154 of 719 LLMsfrom 28 benchmark results across 15 sources

$ how it compares

The LLMs ranked either side of Claude Haiku 4.5, out of 828 with a score, then more from Anthropic.

#	model	score	where	RAM	released
174	Mistral Medium 3.5 (128B) Mistral	65±1	api $3.00/M +local	~42GB	2026-04
175	Command A+ (05-2026) Cohere	64±1	api price unknown +local	~71GB	2026-05
176	Claude Haiku 4.5 Anthropic	64±1	api $2.00/M		2025-10
177	Nemotron-Labs-3 Puzzle 75B-A9B NVIDIA	64±1	local	~35GB	2026-06
178	Qwen3-VL 235B-A22B Thinking Alibaba	64±1	local	~111GB	2025-09
	more from Anthropic
2	Claude Opus 5.5 Anthropic	87±1	api $8.00/M		2026-09
4	Claude Fable 5.1 Anthropic	85±1	api $20.00/M		2026-09
5	Claude Sonnet 5.5 Anthropic	84±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #124 of 300 · 32% of its score · 10 tests

Beats o3, Gemini 2.5 Pro, Gemini 3.1 Flash-Lite here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Large, Nova 2 Pro (Preview) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Multilingual lab	67.4% ✎	▃··▃·▅▃▃█··▃ beats 0/9	hard · avg 31%	▇▇▇▇ 11%	at its level
Chartography with tools lab	8.8% ✎	▃········▃██ beats 0/7	very hard · avg 4.7%	▇▇ 5.8%	sharp · at its level
Terminal-Bench 2.0 with Terminus 2 agent (benchmark authors' runs)	28.3% ✎	▃··▃·▃▃██▅▃▅ beats 7/14	very hard · avg 5.8%	▇▇ 4.5%	sharp
Humanity's Last Exam with tools lab	18.7% ✎	▃······▁·▄·█ beats 0/10	very hard · avg 14%	▇ 3.2%	at its level
BFCL V4 (December 2025 archive)	68.7% ✎	▁▁▁▂██▅▅▆▂▃▂ beats 73/77	hard · avg 32%	▇ 2.8%	at its level
Terminal-Bench 4.0 lab	0.0% ✎	▄·····▄··███ beats 0/7	very hard · avg 0.5%	▇ 1.9%	sharp · too hard for it
Vibe Code Bench (test split, 50 web apps)	11.4% ✎	▃··▃·▃██▃▃▃█ beats 2/14	very hard · avg 2.3%	▇ 1.7%	sharp · noisy
τ²-Bench-Verified (GPT-5.1 user)	53.4% ✎	▂··▄···▂▄▄█▄ beats 1/13	medium · avg 45%	▇ 1.2%	at its level

2 more tests

test	result	vs the others in its run	difficulty	counts	why
KlinikeBench strict pass@1 (333 interactive clinical encounters, Terminus-2)	2.7% ✎	▁▁·▁▃▁▁▅▃█▅█ beats 3/27	very hard · avg 2.4%	▇ 0.8%	too hard for it
T1-Bench multi-domain conversational agent pass^3	34.3% ✎	▂▂··▂·▂·▂▂█▄ beats 7/11	hard · avg 25%	▇ 0.0%	blunt · at its level · noisy

coding▁▁▁▃▆██▇▆▄▃▁ #119 of 458 · 27% of its score · 4 tests

Beats Qwen3.5 Plus, Gemini 2.5 Pro, Kimi K2 Thinking here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Multilingual lab	67.4% ✎	▃··▃·▅▃▃█··▃ beats 0/9	hard · avg 31%	▇▇▇▇ 11%	at its level
Arena WebDev (Code Arena)	1329.4 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 23/119	avg 981	▇▇▇▇ 10%	sharp
Luau Leetcode fully solved (Torpedo Software)	59.6% ✎	▁▁▁▁·▁▂▃▅▃█▇ beats 15/49	hard · avg 18%	▇▇ 5.0%	at its level
Vibe Code Bench (test split, 50 web apps)	11.4% ✎	▃··▃·▃██▃▃▃█ beats 2/14	very hard · avg 2.3%	▇ 1.7%	sharp · noisy

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #94 of 393 * · 5.6% of its score · 1 test

Beats LongCat 2.0 (1.6T-A48B), Inkling Small (266B-A12B), GPT-5.4 nano here, though they rank higher overall. Trails Gemma 4 12B, Kimi K2 Instruct (0905), Gemma 4 26B-A4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1413.8 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 261/379	avg 1268	▇▇ 5.6%	blunt

math▁▂▃▅▆▅▄█▄▁▁▁ #106 of 398 * · 11% of its score · 2 tests

Beats Claude Sonnet 4.6, MiniMax-M3, Nemotron 3 Ultra 550B-A55B here, though they rank higher overall. Trails Gemini 2.5 Flash, Qwen3-VL 32B Thinking, gpt-oss-120b here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH level 5	96.4% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 84/92	hard · avg 32%	▇▇ 5.6%	sharp
OTIS mock AIME 2024–2025	66.7% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 95/193	very hard · avg 8.5%	▇▇ 5.2%	sharp · at its level · noisy

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #61 of 147 * · 1.9% of its score · 1 test

Beats DeepSeek V4 Flash Vision (Exp), GPT-5, Inkling Small (266B-A12B) here, though they rank higher overall. Trails Gemma 4 26B-A4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena document (style controlled)	1418.7 ✎	▁▁▃▃▄▃▇█▅▇▄▅ beats 2/37	avg 1335	▇ 1.9%	blunt

web search▃▁▂▆▄█▃▃▁▁·▁ #79 of 80 * · 3.2% of its score · 1 test

Trails GPT-4.1 nano, GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Humanity's Last Exam with tools lab	18.7% ✎	▃······▁·▄·█ beats 0/10	very hard · avg 14%	▇ 3.2%	at its level

other tests12 not in a task · 34% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond	71.2% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 107/208	hard · avg 40%	▇▇▇▇ 10%	sharp · at its level
ARC-AGI-1 public evaluation set (ARC Prize model_baseline, pass@2)	62.9% ✎	▁·▇▃▁·█▇▁·▇▃ beats 14/27	very hard · avg 4.1%	▇▇ 4.6%	sharp · at its level · noisy
Humanity's Last Exam lab	10.2% ✎	▃·····▂·▄▂█▄ beats 0/10	very hard · avg 5.8%	▇▇ 4.5%	typical
The Token Games Elo (10-model puzzle-duel tournament)	1000.0 ✎	█··▄█··▄·▄·▄ beats 0/7	avg 815	▇ 3.3%	typical
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	65.8% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 52/66	hard · avg 31%	▇ 2.8%	at its level
RobloxQA v2.0 multiple choice (Torpedo Software)	89.5% ✎	▁▁▃▁█▄▃▃▆▂▂▂ beats 12/49	easy · avg 78%	▇ 2.3%	typical
ARC-AGI-2 public evaluation set (ARC Prize model_baseline, pass@2)	5.1% ✎	█··▂▅▁▃▁▃·▁▃ beats 10/24	very hard · avg 0.3%	▇ 1.8%	sharp · too hard for it · noisy
Icelandic LLM Leaderboard average (Miðeind, Sept 2026 suite)	15.7% ✎	▁▁▃▂▂▄▁▃▄▄▄█ beats 4/38	very hard · avg 14%	▇ 1.7%	at its level

4 more tests

test	result	vs the others in its run	difficulty	counts	why
SimpleQA Verified	12.6% ✎	▁▁▃▂▃▃▆▆█▃▂▅ beats 6/78	very hard · avg 8.0%	▇ 1.7%	typical
Chess Puzzles (Epoch)	8.0% ✎	█▁▁▃▁▂▁▄▂▂▁▁ beats 53/135	very hard · avg 5.9%	▇ 1.6%	sharp · too hard for it · noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	95.5% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 77/140	very easy · avg 94%	▇ 0.2%	blunt · maxed out · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	90.2% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 45/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · $2.00/M · $1 in / $5 out per 1M tokens · 200k context · docs.anthropic.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

$ how the score was worked out · Anthropic system cards and mo… 26%, Epoch AI — internal evaluatio… 24%, LMArena leaderboard dataset 17% and 12 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Anthropic system cards and model announcements (lab-reported scores) · lab-reported	5	▇▇▇▇▇▇▇▇ 26%	Factual vendor-reported score (not licensed content)
Epoch AI — internal evaluations	5	▇▇▇▇▇▇▇ 24%	CC BY 4.0
LMArena leaderboard dataset	3	▇▇▇▇▇ 17%	CC BY 4.0
Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode	2	▇▇ 7%	MIT
ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets	2	▇▇ 6%	MIT
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials	1	▇ 4%	CC BY 4.0
The Token Games: Evaluating Language Model Reasoning with Puzzle Duels — authors' own tournament	1	▇ 3%	CC BY-SA 4.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	1	▇ 3%	Apache-2.0
Berkeley Function Calling Leaderboard result archive	1	▇ 3%	Apache-2.0
Vibe Code Bench (Vals AI, arXiv:2603.04601v3)	1	▇ 2%	CC-BY-4.0
Miðeind Icelandic LLM Leaderboard	1	▇ 2%	CC BY 4.0
Amazon AGI — τ²-Bench-Verified	1	▇ 1%	MIT
KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)	1	▇ 1%	CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	2	▇ 0%	Apache-2.0
T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)	1	▇ 0%	CC BY 4.0
prior · pull toward the middle		▇ 0%
weighted average		= 64.2