whatmodel.sh / Mistral Medium 3.1 (2508)
Mistral Medium 3.1 (2508)
↗ share
+ contribute
Mistral Medium 3.1 (2508) is a closed-weights language model from Mistral, released August 2025.
Catalogue identity notes
Comparison-only. Arena entry 'mistral-medium-2508' (text, vision). Released 2025-08: Mistral model card: 'released August 2025'. Mistral card status 'Deprecated' (deprecation date 5/22/2026). Mistral Medium 3 (2505), deprecated the same day, is recorded in the catalogue as retired 8/31/2026, so no current API is asserted.
$ how it compares
The LLMs ranked either side of Mistral Medium 3.1 (2508), out of 828 with a score, then more from Mistral.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 178 |  Qwen3-VL 235B-A22B Thinking | 64±1 | local | ||
| 179 | Qwen3-Coder 480B-A35B Instruct | 64±1 | api $0.67/M +local | ||
| 180 | Mistral Medium 3.1 (2508) | 64±2 | comparison only | ||
| 181 |  Gemma 4 26B-A4B | 64±1 | api $0.14/M +local | ||
| 182 |  Kimi K2 Instruct (0905) | 64±2 | local | ||
| more from Mistral | |||||
| 79 | Mistral Large 4 (public preview) | 71±2 | api $2.06/M | ||
| 174 | Mistral Medium 3.5 (128B) | 65±1 | api $3.00/M +local | ||
| 200 | Magistral Medium 1.2 (2509) | 63±2 | comparison only | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #113 of 340 * · 11% of its score · 1 test
Beats Grok 4.3, o3-pro, Qwen3.6 Plus here, though they rank higher overall. Trails SmolLM3 3B, Nova 2 Pro (Preview) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Galileo Agent Leaderboard v2 (action completion) | 61.0% ✎ | ▅▃·▃▃▅▅█▃█▅▅ beats 18/19 | 11% |
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #116 of 463 * · 50% of its score · 1 test
Beats LongCat 2.0 (1.6T-A48B), Inkling Small (266B-A12B), GPT-5.4 nano here, though they rank higher overall. Trails Mistral Large 3 (675B-A41B), GPT-4.1, Gemma 4 12B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| LMArena (text) | 1407.9 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 252/379 | 50% |
vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #112 of 182 * · 35% of its score · 1 test
Beats DeepSeek V4 Flash Vision (Exp) here, though they rank higher overall. Trails GPT-4o (2024-11-20), Mistral Small 4 (119B-A6.5B), Qwen3-VL 30B-A3B Thinking here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena vision (style controlled) | 1159.4 ✎ | ▁▁▂▂▂▂▄▅▅▆██ beats 53/145 | 35% |
other tests1 not in a task · 1.7% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 88.6% ✎ | ▁▁▂▃▄▄█▆▃▂▂▁ beats 18/140 | 1.7% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · LMArena leaderboard dataset 85%, Galileo Agent Leaderboard (v1… 11%, Vectara hallucination leaderb… 2%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 LMArena leaderboard dataset | 2 | 85% |
 Galileo Agent Leaderboard (v1 and v2 results) | 1 | 11% |
 Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository) | 1 | 2% |
| prior · pull toward the middle | 3% | |
| weighted average | = 64.0 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json