whatmodel.sh / Claude 2.1
Claude 2.1
↗ share
+ contribute
Claude 2.1 is a closed-weights language model from Anthropic, released November 2023. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2023-11-21). API model claude-2.1. Anthropic model deprecations page: retired July 21, 2025.
$ how it compares
The LLMs ranked either side of Claude 2.1, out of 828 with a score, then more from Anthropic.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 529 |  DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) | 45±1 | local | ||
| 530 |  GPT-4o Audio Preview (2024-10-01) | 45±3 | comparison only | ||
| 531 | Claude 2.1 | 45±1 | comparison only | ||
| 532 |  Zephyr ORPO 141B-A35B v0.1 | 45±2 | local | ||
| 533 |  Phi-3-medium-4k-instruct | 45±1 | local | ||
| more from Anthropic | |||||
| 2 | Claude Opus 5.5 | 87±1 | api $8.00/M | ||
| 4 | Claude Fable 5.1 | 85±1 | api $20.00/M | ||
| 5 | Claude Sonnet 5.5 | 84±1 | api $4.00/M | ||
$ what it's good at
coding▁▁▁▃▅▇█▇▅▄▃▁ #366 of 524 * · 29% of its score · 1 test
Beats Solar Open 100B (102B-A12B), GLM-4.7-Flash, Qwen3 4B Instruct 2507 here, though they rank higher overall. Trails DeepSeek LLM 67B Chat, WizardLM 70B V1.0, Phi-4-mini 3.8B here, though they rank lower overall.
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #358 of 463 · 47% of its score · 5 tests
Beats Ministral 3 14B Reasoning 2512, Gemma 4 E4B, OpenThinker3 7B here, though they rank higher overall. Trails LFM2 350M, Flan-T5 XXL, LFM2 700M here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AlpacaEval 2 length-controlled | 25.3% ✎ | ▁▂▂▃▆▆█▆▄▄▁▁ beats 64/122 | 21% |
| Arena-Hard (v0.1, GPT-4-1106 judge) | 22.8% ✎ | ▂▁▁▅█▅▄▃▃▅▃▁ beats 17/60 | 14% |
| Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024) | 1156.0 ✎ | ▂▁▂▁▃▇█▆▇▄▂▂ beats 69/95 | 6.0% |
| MT-Bench (GPT-4 single-answer grading, LMSYS runs) | 8.1800 MT-Bench score (1-10) ✎ | ▆▂▂▆▆▄██▂▄▄▆ beats 23/28 | 5.3% |
| AlpacaEval 1.0 win rate (vs text-davinci-003, GPT-4 judge) | 87.1% ✎ | ▂▂·▆·▇▃▆█▆▄▂ beats 25/45 | 1.1% |
math▁▂▃▅▇▅▅█▄▁▁▁ #344 of 462 * · 0.7% of its score · 1 test
Beats GPT-4 (0613), Athene-70B, Gemma 2 27B IT here, though they rank higher overall. Trails Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B, CodeGemma 7B (v1.0) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| OTIS mock AIME 2024–2025 | 1.9% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 16/193 | 0.7% |
other tests2 not in a task · 23% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| EvoEval (overall) | 46.2% ✎ | ▂▂▃▄▄█▁▄▅▃▁▃ beats 31/42 | 19% |
| GPQA Diamond | 33.0% ✎ | ▁▁▁▁▃▄▆▄▅▇█▅ beats 23/208 | 4.3% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Top Leaderboard Ranking = Top… 47%, AlpacaEval 1.0 and 2.0 leader… 22%, Arena-Hard-Auto v0.1 leaderbo… 14% and 3 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json