whatmodel.sh / Claude Instant 1.2

Claude Instant 1.2

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Claude Instant 1.2 is a closed-weights language model from Anthropic, released August 2023. Licence: proprietary.

Catalogue identity notes

Comparison-only: retired or no longer listed API model, kept so its independent runs can be scored. Anthropic API model claude-instant-1.2 (announced in 'Releasing Claude Instant 1.2', anthropic.com/news/releasing-claude-instant-1-2, the link EvalPlus prints for its Claude Instant entry). Anthropic model deprecations page: claude-instant-1.2 retired November 6, 2024. Distinct from Claude 2.x and Claude 3 Haiku.

42.1±0.8comparison only: no current run option verifiedfrom 3 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of Claude Instant 1.2, out of 828 with a score, then more from Anthropic.

#modelscorewhereRAMreleased
587Mathstral 7B v0.1 Mistral AI42±1local~5.8GB2024-07
588Yi-1.5 9B (base) 01.AI42±2local~6.8GB2024-05
589Claude Instant 1.2 Anthropic42±1comparison only2023-08
590Llama 3 8B Instruct Meta42±0local~6.6GB2024-04
591WizardLM 70B V1.0 Microsoft42±1local~32GB2023-08
more from Anthropic
2Claude Opus 5.5 Anthropic87±1api $8.00/M2026-09
4Claude Fable 5.1 Anthropic85±1api $20.00/M2026-09
5Claude Sonnet 5.5 Anthropic84±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▅▇█▇▅▄▃▁ #413 of 524 * · 85% of its score · 2 tests

Beats LFM2.5 8B-A1B, Qwen3-VL 2B Thinking, DeepSeek R1 Distill Qwen 7B here, though they rank higher overall. Trails Phi-3-mini-128k-instruct (Apr 2024), Code Tulu 2 13B, DeepSeek LLM 67B Chat here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval57.3% ✎▁·▁▁▃▄▆▅█▃▁▁ beats 45/99easy · avg 79%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 48%sharp · at its level
HumanEval+50.6% ✎▁·▁▁▁▃▄▄▄█▃▁ beats 41/99easy · avg 74%▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 36%sharp · at its level
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #348 of 463 * · 15% of its score · 1 test

Beats Ministral 3 14B Reasoning 2512, Gemma 4 E4B, OpenThinker3 7B here, though they rank higher overall. Trails Flan-T5 XXL, LFM2 700M, EXAONE 3.5 2.4B Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
AlpacaEval 2 length-controlled25.6% ✎▁▂▂▃▆▆█▆▄▄▁▁ beats 68/122medium · avg 39%▇▇▇▇▇▇ 15%typical

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · EvalPlus / EvalPerf published… 85%, AlpacaEval 1.0 and 2.0 leader… 15%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
EvalPlus / EvalPerf published evaluation runs2▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 85%Apache-2.0
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)1▇▇▇▇▇ 15%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 42.1

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json