whatmodel.sh / Claude Instant 1.1

Claude Instant 1.1

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Claude Instant 1.1 is a closed-weights language model from Anthropic, released May 2023. Licence: proprietary.

Catalogue identity notes

Anthropic API model claude-instant-v1.1 (later id claude-instant-1.1). Absent from the archived API reference of 2023-03-15; the 2023-05-11 snapshot lists it as 'Our latest version of claude-instant-v1'. Launch date not published; 'released' is the month first documented (true launch between 2023-03-15 and 2023-05-11). Retired 2024-11-06. Comparison-only.

Model information sources

Release date ↗

44.7±1.6comparison only: no current run option verifiedfrom 6 benchmark results across 2 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of Claude Instant 1.1, out of 877 with a score, then more from Anthropic.

#modelscorewhereRAMreleased
577Code Llama 70B (base) Meta45±1local~28GB2024-01
578Qwen3-VL 2B Instruct Alibaba45±1local~2.7GB2025-10
579Claude Instant 1.1 Anthropic45±2comparison only2023-05
580Reka Flash 21B (2024-02-26) Reka AI45±1comparison only2024-02
581LFM2.5 1.2B Instruct Liquid AI45±2local~1.3GB2026-01
more from Anthropic
2Claude Opus 5.5 Anthropic85±1api $8.00/M2026-09
4Claude Fable 5.1 Anthropic83±1api $20.00/M2026-09
6Claude Sonnet 5.5 Anthropic82±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▅▇█▇▅▄▂▁ #422 of 529 * · 20% of its score · 1 test

Beats LFM2.5 8B-A1B, Qwen3-VL 2B Thinking, DeepSeek R1 Distill Qwen 7B here, though they rank higher overall. Trails Qwen2.5 1.5B Instruct, Code Tulu 2 13B, Phi-3-mini-128k-instruct (Apr 2024) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval lab +152.8% ✎▁▁▁▁▂▂▆▆█▅▂▁ beats 99/205easy · avg 71%▇▇▇▇▇▇▇▇ 20%sharp · at its level
math▁▂▃▆█▇▅▇▃▁▁▁ #371 of 523 * · 29% of its score · 1 test

Beats Molmo 2 8B, Llama 3.1 Nemotron 70B Instruct, GPT-4 (0613) here, though they rank higher overall. Trails StarCoder2 7B, DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 7B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K lab +180.9% ✎▂▂▂▂▃▄▅▇██▆▄ beats 142/200very easy · avg 90%▇▇▇▇▇▇▇▇▇▇▇▇ 29%sharp
other tests4 not in a task · 50% of its score
testresultvs the others in its rundifficultycountswhy
ARC Challenge lab85.7% ✎▄▁▁▂▁▄█▇▃▂▃▂ beats 133/140very easy · avg 90%▇▇▇▇▇▇▇▇▇ 21%sharp · small run
MMLU lab73.4% ✎▁▁▁▁▁▂▂▅██▆▄ beats 150/214easy · avg 80%▇▇▇▇▇▇▇ 18%at its level · small run
TriviaQA lab78.9% ✎▁▁▂▂▄▄▃▆▅█▅▂ beats 76/80easy · avg 84%▇▇▇ 7.8%small run
ConvRe Re2Text, normal relation, zero-shot (prompt 1#)65.7% ✎▂▂▄··▃▂█▄··▂ beats 4/12easy · avg 90%▇ 2.6%at its level · noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Anthropic system cards and mo… 96%, An Investigation of LLMs' Ine… 3%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Anthropic system cards and model announcements (lab-reported scores) · lab-reported7Factual vendor-reported score (not licensed content)
An Investigation of LLMs' Inefficacy in Understanding Converse Relations — authors' own runs1CC BY 4.0
prior · pull toward the middle
weighted average= 44.7

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json