whatmodel.sh / Ministral 3B (2410)

Ministral 3B (2410)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Ministral 3B (2410) is a closed-weights language model from Mistral, released October 2024. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-10-16). API model ministral-3b-2410 ('Ministral 3B', version 24.10; retired 12/31/2025); Epoch model_metadata: 'API access'. Distinct from Ministral 3 3B (2512).

41.5±0.8comparison only: no current run option verifiedfrom 24 benchmark results across 4 sources

$ how it compares

The LLMs ranked either side of Ministral 3B (2410), out of 828 with a score, then more from Mistral.

#modelscorewhereRAMreleased
596speechless-starcoder2-7b uukuguy42±1local~6.1GB2024-03
597Command R (03-2024) Cohere42±1local~24GB2024-03
598Ministral 3B (2410) Mistral42±1comparison only2024-10
599OLMo 2 7B Instruct (1124) Allen Institute for AI42±1local~5.8GB2024-12
600WizardCoder-15B-V1.0 WizardLM42±1local~12GB2023-06
more from Mistral
79Mistral Large 4 (public preview) Mistral71±2api $2.06/M2026-10
174Mistral Medium 3.5 (128B) Mistral65±1api $3.00/M +local~42GB2026-04
180Mistral Medium 3.1 (2508) Mistral64±2comparison only2025-08

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▄··········█ #2 of 3 * · 4.7% of its score · 2 tests
testresultvs the others in its rundifficultycountswhy
HumanEval lab77.4% ✎▁▁▁▁▁▃▅▅█▃▂▁ beats 153/190easy · avg 83%▇ 2.6%sharp · at its level · small run
MBPP lab67.7% ✎▁▁▁▄▅█▄▅▃▁▂▁ beats 61/81easy · avg 76%▇ 2.2%at its level · small run
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #287 of 463 * · 1.9% of its score · 1 test

Beats Mistral Small 4 (119B-A6.5B), Ministral 3 14B Reasoning 2512, Qwen2.5 Coder 32B Instruct here, though they rank higher overall. Trails Flan-T5 XXL, Granite 4.0 H 1B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Arena-Hard (version/judge unspecified) lab64.3% ✎▃▃▂▅▃▆▅█▅▅▂▃ beats 14/30easy · avg 70%▇ 1.9%at its level · small run
math▁▂▃▅▇▅▅█▄▁▁▁ #391 of 462 · 27% of its score · 5 tests

Beats GPT-3.5 Turbo (0125), Qwen2-VL 7B Instruct, DBRX Instruct here, though they rank higher overall. Trails Qwen3 0.6B, Gemma 7B (base), LFM2 700M here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K80.1% ✎█·█▄▄█···█·▄ beats 2/10very easy · avg 91%▇▇▇▇ 10%sharp
MGSM44.6% ✎▅▃▃█·▅···▃·▃ beats 0/10easy · avg 71%▇▇▇ 6.8%at its level
MATH level 514.4% ✎▂▂▅▆▄█▆▃▅▂▃▄ beats 11/92hard · avg 32%▇▇ 4.3%sharp
MATH41.8% ✎█▄██···▄▄▄·▄ beats 1/10medium · avg 64%▇▇ 4.1%at its level
MATH (subset unspecified) lab51.7% ✎▂▄▄▃█▇▅▆▄▃▄▁ beats 28/80easy · avg 66%▇ 1.6%at its level · noisy
other tests16 not in a task · 66% of its score
testresultvs the others in its rundifficultycountswhy
ARC Challenge80.3% ✎▂▂▃█▂···▄··▂ beats 2/10very easy · avg 91%▇▇▇▇▇▇ 15%sharp · at its level
MMLU60.8% ✎█▄█▄▄▄·▄▄··▄ beats 0/10easy · avg 77%▇▇▇ 8.4%at its level
OpenBookQA79.8% ✎▂··▄█▂·▂···▄ beats 3/10easy · avg 87%▇▇▇ 8.1%at its level
BIG-Bench Hard51.2% ✎██▄·█▄·▄▄··▄ beats 0/10easy · avg 74%▇▇▇ 7.6%sharp · at its level
MMLU-Pro35.3% ✎█▄▄·█▄▄▄·▄·▄ beats 0/10medium · avg 57%▇▇ 4.9%at its level
Multilingual MMLU (Microsoft)46.4% ✎█▄▄██··█···▄ beats 0/10easy · avg 65%▇▇ 4.8%at its level
HellaSwag74.6% ✎▃▅█·▃▃▃···▃▃ beats 4/10easy · avg 84%▇ 3.0%at its level
WinoGrande59.8% ✎▃····▃▃█▃·▅▅ beats 1/10medium · avg 74%▇ 2.9%at its level
8 more tests
testresultvs the others in its rundifficultycountswhy
BoolQ79.4% ✎▃··▃··██▃·▃▃ beats 4/10easy · avg 85%▇ 2.8%at its level
GPQA Diamond25.3% ✎▃▁▁▁▂▄▆▄▅▇█▅ beats 0/208hard · avg 40%▇ 2.8%sharp · too hard for it
Social IQA73.9% ✎▃·▃▅▅█·▃···▃ beats 4/10easy · avg 78%▇ 2.6%blunt · at its level
GPQA (subset unspecified)24.4% ✎█··▄▂··▂▂··▄ beats 0/10very hard · avg 33%▇ 1.1%sharp · too hard for it · noisy
TruthfulQA MC262.9% ✎▂·█▂▂▄····▂▂ beats 1/10easy · avg 69%▇ 0.9%blunt · at its level
PIQA73.2% ✎▃·▅·▃█·▃▅··▃ beats 2/10easy · avg 83%▇ 0.9%blunt · at its level
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)91.7% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 35/140very easy · avg 94%▇ 0.5%blunt · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)92.7% ✎▁▂▁▂▅█▂▃▄▃▁▁ beats 68/98very easy · avg 90%▇ 0.0%blunt

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · microsoft/Phi-4-mini-instruct… 84%, Mistral AI vendor-reported be… 8%, Epoch AI — internal evaluatio… 7% and 1 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
microsoft/Phi-4-mini-instruct model card16▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 84%MIT
Mistral AI vendor-reported benchmark scores for its own models (Hugging Face model cards) · lab-reported4▇▇ 8%Factual vendor-reported score (not licensed content)
Epoch AI — internal evaluations2▇▇ 7%CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)2▇ 0%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 41.5

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json