whatmodel.sh / Mistral Medium (2312)
Mistral Medium (2312)
↗ share
+ contribute
Mistral Medium (2312) is a closed-weights language model from Mistral, released December 2023.
Catalogue identity notes
Comparison-only. Arena entry 'mistral-medium' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'mistral-medium' as 'Mistral Medium' (Mistral, Proprietary), linking https://mistral.ai/news/la-plateforme/. Released 2023-12: Mistral model card 'Mistral Medium 1.0' dated December 11, 2023 (API name mistral-medium-2312). Arena key 'mistral-medium'; LMArena's table labels it 'Mistral Medium' linking mistral.ai/news/la-plateforme (Dec 2023). Mistral card: retired 6/16/2025.
$ how it compares
The LLMs ranked either side of Mistral Medium (2312), out of 828 with a score, then more from Mistral.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 511 |  WizardCoder Python 34B V1.0 | 46±1 | local | ||
| 512 |  Gemini 1.0 Pro (001) | 46±2 | comparison only | ||
| 513 | Mistral Medium (2312) | 46±1 | comparison only | ||
| 514 |  GPT-3.5 Turbo (0613) | 46±1 | comparison only | ||
| 515 |  Granite 3.3 8B Instruct | 46±1 | local | ||
| more from Mistral | |||||
| 79 | Mistral Large 4 (public preview) | 71±2 | api $2.06/M | ||
| 174 | Mistral Medium 3.5 (128B) | 65±1 | api $3.00/M +local | ||
| 180 | Mistral Medium 3.1 (2508) | 64±2 | comparison only | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #283 of 340 * · 2.6% of its score · 1 test
Beats DeepSeek R1 0528 Qwen3 8B, Magistral Small 1.0 (2506), Llama 4 Scout (109B-A17B) here, though they rank higher overall. Trails Granite 4.0 H 1B, Granite 4.0 1B, SmolLM3 3B here, though they rank lower overall.
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #344 of 463 · 97% of its score · 5 tests
Beats Ministral 3 14B Reasoning 2512, Gemma 4 E4B, OpenThinker3 7B here, though they rank higher overall. Trails Flan-T5 XXL, LFM2 700M, EXAONE 3.5 2.4B Instruct here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AlpacaEval 2 length-controlled | 28.6% ✎ | ▁▂▂▃▆▆█▆▄▄▁▁ beats 76/122 | 42% |
| Arena-Hard (v0.1, GPT-4-1106 judge) | 31.9% ✎ | ▂▁▁▅█▅▄▃▃▅▃▁ beats 27/60 | 29% |
| Polyrating rating (bias-aware re-fit of Chatbot Arena votes, mid-2024) | 1153.0 ✎ | ▂▁▂▁▃▇█▆▇▄▂▂ beats 68/95 | 12% |
| LMArena (text) | 1222.7 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 77/379 | 12% |
| AlpacaEval 1.0 win rate (vs text-davinci-003, GPT-4 judge) | 96.8% ✎ | ▂▂·▆·▇▃▆█▆▄▃ beats 44/45 | 1.9% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · AlpacaEval 1.0 and 2.0 leader… 44%, Arena-Hard-Auto v0.1 leaderbo… 29%, Polyrating: A Cost-Effective… 12% and 2 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository) | 2 | 44% |
| Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2) | 1 | 29% |
 Polyrating: A Cost-Effective and Bias-Aware Rating System for LLM Evaluation (arXiv:2409.00696v3) | 1 | 12% |
 LMArena leaderboard dataset | 1 | 12% |
 Berkeley Function Calling Leaderboard result archive | 1 | 3% |
| prior · pull toward the middle | 1% | |
| weighted average | = 45.7 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json