whatmodel.sh / Mistral Small 3.2 24B

Mistral Small 3.2 24B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Mistral Small 3.2 24B is an open-weights language model from Mistral, released June 2025. Its published size is 24B parameters and a 128k-token context window. Licence: apache-2.0.

55.8±1.0#249 of 719 LLMsfrom 23 benchmark results across 10 sources

$ how it compares

The LLMs ranked either side of Mistral Small 3.2 24B, out of 828 with a score, then more from Mistral.

#modelscorewhereRAMreleased
298Qwen2.5-VL 72B Instruct Alibaba56±1local~46GB2025-01
299Granite 4.1 30B IBM56±1local~19GB2026-04
300Mistral Small 3.2 24B Mistral56±1api $0.11/M +local~16GB2025-06
301DeepSeek-V2.5 (1210) DeepSeek56±1local~82GB2024-12
302Qwen3 8B Alibaba56±1api $0.31/M +local~6.1GB2025-04
more from Mistral
79Mistral Large 4 (public preview) Mistral71±2api $2.06/M2026-10
174Mistral Medium 3.5 (128B) Mistral65±1api $3.00/M +local~42GB2026-04
180Mistral Medium 3.1 (2508) Mistral64±2comparison only2025-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #211 of 300 * · 6.2% of its score · 2 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 (Jan '25) here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Granite 4.1 3B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
BFCL V4 (December 2025 archive)37.1% ✎▁▁▁▂██▅▅▆▂▂▂ beats 41/77hard · avg 32%▇▇ 4.5%at its level
Galileo Agent Leaderboard v2 (action completion)26.0% ✎▅▃·▃▃▅▅█▃█▅▅ beats 3/19hard · avg 27%▇ 1.8%at its level
codingnot ranked on this task yet · 2 tests
testresultvs the others in its rundifficultycountswhy
MBPP+ pass@5 lab78.3% ✎█··········█ beats 1/1—not counted yet: no other model in the same run to compare it with
HumanEval+ pass@5 lab92.9% ✎█··········█ beats 1/1—not counted yet: no other model in the same run to compare it with
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #152 of 393 * · 14% of its score · 2 tests

Beats INTELLECT-3, MiniMax-M2, MiniCPM5 2B here, though they rank higher overall. Trails LFM2.5 1.2B Thinking, LFM2.5 8B-A1B, LFM2.5 2.6B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1356.5 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 201/379avg 1268▇▇▇▇ 9.2%blunt
Arena-Hard v2 aggregate lab43.1% ✎▁▁··▄▃▁▃█▁▁▇ beats 5/22hard · avg 22%▇▇ 4.5%sharp · at its level · small run
math▁▂▃▅▆▅▄█▄▁▁▁ #199 of 398 * · 11% of its score · 2 tests

Beats MiniMax-M2.1, Qwen3 235B-A22B (Apr 2025, hybrid), DeepSeek V3 (Dec '24) here, though they rank higher overall. Trails Phi-4-mini-reasoning 3.8B, DeepSeek R1 Distill Qwen 7B, OpenThinker2 7B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
OTIS mock AIME 2024–202530.3% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 65/193very hard · avg 8.5%▇▇▇ 7.0%sharp · noisy
MATH (subset unspecified) lab69.4% ✎▂▄▄▃█▇▅▆▄▃▄▁ beats 43/80medium · avg 63%▇▇ 4.1%at its level · small run
vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #101 of 147 · 17% of its score · 6 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-4o (2024-08-06), Qwen3-VL 4B Thinking here, though they rank higher overall. Trails Llama 4 Scout (109B-A17B) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Arena vision (style controlled)1139.8 ✎▁▁▂▂▂▂▄▅▅▆██ beats 42/145avg 1068▇▇▇ 6.4%blunt
MMMU Pro lab49.1% ✎▂▃▁▃▆▇▃▇▇██▅ beats 9/52hard · avg 42%▇ 3.2%at its level · small run
MMMU (val) lab62.5% ✎▁····▁▁▄█▇▆▆ beats 24/52medium · avg 56%▇ 2.9%at its level · small run
VMMU (Vietnamese multimodal understanding and reasoning, mean accuracy over 7 tasks)31.9% ✎▅······▃·▅·█ beats 2/7very hard · avg 36%▇ 2.4%noisy
DocVQA lab94.9% ✎▁··▃▁▁▁▇█▇▄▅ beats 16/28very easy · avg 93%▇ 1.0%blunt · maxed out · small run
ChartQA lab87.4% ✎▁··▁▁▁▁▁▂▅▅█ beats 32/44easy · avg 82%▇ 0.9%blunt · small run
other tests12 not in a task · 52% of its score
testresultvs the others in its rundifficultycountswhy
GPQA Diamond +149.1% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 67/208hard · avg 40%▇▇▇▇▇▇▇▇▇ 23%sharp · at its level · several runs
EQ-Bench 3 (emotional intelligence, judge Elo)1162.3 ✎▁·▂▁▄▅▅▃█▄▁▁ beats 23/62avg 1044▇▇ 5.7%typical
CPTUB average (SpeakLeash, 0-5 judge scale)76.5% ✎▁▂▂▂▃█▃▆▅▄▂▃ beats 74/95easy · avg 66%▇▇ 5.4%at its level
MMLU lab80.5% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 138/183easy · avg 74%▇▇ 4.3%small run
SimpleQA lab12.1% ✎▄▄▄▆█▆███·▂█ beats 14/32very hard · avg 7.9%▇▇ 3.8%small run
MMLU-Pro lab69.1% ✎▁▁·▁▁▁▃▂▄▅█▁ beats 83/188medium · avg 58%▇ 3.7%at its level · small run
IFBench (aggregation unspecified) lab34.0% ✎▃▇▂▄▅▅▂▄▄▆█▇ beats 19/80hard · avg 31%▇ 2.7%at its level · small run
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy90.4% ✎▃·▁▁▂▁▂▅█▇▄▁ beats 92/100medium · avg 68%▇ 1.4%at its level · noisy
4 more tests
testresultvs the others in its rundifficultycountswhy
Chess Puzzles (Epoch)1.0% ✎█▁▁▁▁▂▁▄▂▂▁▁ beats 39/135very hard · avg 5.9%▇ 0.8%sharp · too hard for it · noisy
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy95.1% ✎▁▁▁▁▁▁▁▂▄█▄▃ beats 141/146easy · avg 83%▇ 0.8%noisy
GPQA Main lab44.2% ✎█··········█ beats 0/1—not counted yet: no other model in the same run to compare it with
WildBench v2 lab65.3% ✎█··········█ beats 1/1—not counted yet: no other model in the same run to compare it with

$ where to run it

Through an API

api · DeepInfra · $0.11/M · $0.075 in / $0.2 out per 1M tokens · 128k context · deepinfra.com ↗ · price sourceUSD per 1M text tokens, standard uncached input and output. Product: Serverless text generation. API model: mistralai/Mistral-Small-3.2-24B-Instruct-2506. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD.

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Mistral Small 3.2 24B · needs ~14.3 GB · from huggingface.co/lmstudio-community
    chat in the terminal
    llama cli -hf lmstudio-community/Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf lmstudio-community/Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Mistral Small 3.2 24B · needs ~14.3 GB · from huggingface.co/lmstudio-community

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/lmstudio-community/Mistral-Small-3.2-24B-Instruct-2506-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Mistral Small 3.2 24B · needs ~14.9 GB · from ollama.com
    chat in the terminal
    ollama run mistral-small3.2:24b

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Mistral Small 3.2 24B · needs ~13.5 GB, Mac only · from huggingface.co/lmstudio-community
    chat in the terminal
    mlx_lm.chat --model lmstudio-community/Mistral-Small-3.2-24B-Instruct-2506-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model lmstudio-community/Mistral-Small-3.2-24B-Instruct-2506-MLX-4bit

Weights and documentation: huggingface.co/mistralai ↗

$ how the score was worked out · mistralai/Mistral-Small-3.2-2… 33%, Epoch AI — internal evaluatio… 23%, LMArena leaderboard dataset 16% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
mistralai/Mistral-Small-3.2-24B-Instruct-2506 model card · lab-reported9▇▇▇▇▇▇▇▇▇▇ 33%Apache-2.0
Epoch AI — internal evaluations3▇▇▇▇▇▇▇ 23%CC BY 4.0
LMArena leaderboard dataset2▇▇▇▇▇ 16%CC BY 4.0
mistralai/Mistral-Small-4-119B-2603 model card · lab-reported2▇▇ 6%Apache-2.0
EQ-Bench 3 canonical leaderboard results1▇▇ 6%MIT
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)1▇▇ 5%MIT
Berkeley Function Calling Leaderboard result archive1▇ 4%Apache-2.0
VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark (arXiv:2508.13680v5)1▇ 2%CC-BY-4.0
EuroEval leaderboards (results collection), English generative leaderboard2▇ 2%MIT
Galileo Agent Leaderboard (v1 and v2 results)1▇ 2%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 55.8

not counted (4)

mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardGPQA Main [correct]44.2%fewer than three models have this test yet, or this source ran it for this model alone
mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardHumanEval+ pass@5 [correct]92.9%fewer than three models have this test yet, or this source ran it for this model alone
mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardMBPP+ pass@5 [correct]78.3%fewer than three models have this test yet, or this source ran it for this model alone
mistralai/Mistral-Small-3.2-24B-Instruct-2506 model cardWildBench v2 [correct]65.3%fewer than three models have this test yet, or this source ran it for this model alone
$ quantization evidence · 6 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precisiontestfull → quantizedkept
NVFP4 (FP4 weights and activations, tensor-group size 16; SmoothQuant + QuantizationModifier via llm-compressor)IFEval 0-shot (OpenLLM v2 config)0.841 → 0.844100.4%source ↗
NVFP4 (FP4 weights and activations, tensor-group size 16; SmoothQuant + QuantizationModifier via llm-compressor)OpenLLM v1 average of 6 tasks (ARC-Challenge 25-shot, GSM8K 5-shot, HellaSwag 10-shot, MMLU 5-shot, TruthfulQA MC2 0-shot, Winogrande 5-shot; lm-evaluation-harness `openllm`, vLLM)0.782 → 0.76898.1%source ↗
NVFP4 (FP4 weights and activations, tensor-group size 16; SmoothQuant + QuantizationModifier via llm-compressor)MMLU (OpenLLM v1 config, lm-evaluation-harness)0.811 → 0.79498.0%source ↗
NVFP4 (FP4 weights and activations, tensor-group size 16; SmoothQuant + QuantizationModifier via llm-compressor)OpenLLM v2 average of 6 tasks (IFEval, BBH, MATH-Hard, GPQA, MuSR, MMLU-Pro; lm-evaluation-harness `leaderboard`, chat template, vLLM; values exactly as published, per-task normalisation not stated)0.525 → 0.51297.7%source ↗
NVFP4 (FP4 weights and activations, tensor-group size 16; SmoothQuant + QuantizationModifier via llm-compressor)GSM8K (OpenLLM v1 config, lm-evaluation-harness)0.896 → 0.87197.2%source ↗
NVFP4 (FP4 weights and activations, tensor-group size 16; SmoothQuant + QuantizationModifier via llm-compressor)MMLU-Pro 5-shot (OpenLLM v2 config)0.508 → 0.48495.3%source ↗

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json