whatmodel.sh / Olmo 3 32B Think

Olmo 3 32B Think

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Olmo 3 32B Think is an open-weights language model from Ai2, released November 2025. Its published size is 32.2B parameters and a 66k-token context window. Licence: apache-2.0.

Catalogue identity notes

Arena entry 'olmo-3-32b-think' (text). Maker repo allenai/Olmo-3-32B-Think @f2edda1: licence tag apache-2.0. 32,233,522,176 parameters from the Hugging Face safetensors metadata (dense, so active = total). Released 2025-11: Ai2 Olmo 3 announcement dated November 20, 2025. Olmo 3 (Nov 2025), distinct from the catalogued olmo-3-1-32b-think (3.1).

57.9±1.0#218 of 719 LLMsfrom 18 benchmark results across 4 sources

$ how it compares

The LLMs ranked either side of Olmo 3 32B Think, out of 828 with a score, then more from Ai2.

#modelscorewhereRAMreleased
260DeepSeek R1 Distill Llama 70B DeepSeek58±1api $0.80/M +local~30GB2025-01
261Phi-4-reasoning 14B Microsoft58±1api $0.22/M +local~10GB2025-04
262Olmo 3 32B Think Ai258±1local~21GB2025-11
263Magistral Small 1.2 Mistral58±1api $0.75/M +local~16GB2025-09
264Nemotron-Cascade-8B-Thinking NVIDIA58±2local~6.3GB2025-12
more from Ai2
259Olmo 3.1 32B Think Ai258±1local~21GB2025-12
314Molmo 2 8B Ai255±2local~6.8GB2025-12
326Olmo 3.1 32B Instruct Ai255±2local~21GB2025-12

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▆██▇▆▄▃▁ #205 of 458 · 8.2% of its score · 3 tests

Beats Qwen3.5 4B, Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B) here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 3B Reasoning 2512, DeepSeek-Coder-V2 Instruct (0724) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
MBPP+ lab68.0% ✎▄▄····▄·▄··█ beats 4/5medium · avg 62%▇ 3.1%at its level
HumanEval+ lab91.4% ✎▂·······▂·▂█ beats 5/6easy · avg 80%▇ 2.7%sharp
LiveCodeBench v3 lab83.5% ✎▃▃·····▃·▃·█ beats 6/6easy · avg 65%▇ 2.4%typical
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #211 of 393 · 13% of its score · 3 tests

Beats Granite 4.2 8B, Mistral Small 4 (119B-A6.5B), Gemma 2 9B IT SimPO here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, Granite 4.0 H Micro 3B, Gemma 3n E4B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1306.4 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 135/379avg 1268▇▇▇ 8.2%blunt
AlpacaEval 2 length-controlled lab74.2% ✎█···█·█·█·██ beats 5/5medium · avg 44%▇▇ 4.5%at its level
IFEval (aggregation unspecified) lab89.0% ✎▄▄·····█▄▄·▄ beats 5/6easy · avg 79%▇ 0.5%noisy
math▁▂▃▅▆▅▄█▄▁▁▁ #134 of 398 · 24% of its score · 4 tests

Beats Claude Sonnet 4.6, MiniMax-M3, MiMo-V2.5 (310B-A15B) here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
AIME 2024 lab76.8% ✎▄·····▄··▄██ beats 5/6medium · avg 59%▇▇▇▇ 10%sharp
AIME 2025 lab72.5% ✎▄··▄····█▄▄▄ beats 5/6medium · avg 51%▇▇▇▇ 10%sharp · at its level
OMEGA (math generalisation) lab50.8% ✎▅···▃▃···▃▃█ beats 7/8hard · avg 24%▇ 2.8%at its level
MATH (subset unspecified) lab96.1% ✎▅····▃▃··▃▃█ beats 6/8easy · avg 74%▇ 1.1%noisy
other tests8 not in a task · 54% of its score
testresultvs the others in its rundifficultycountswhy
GPQA (subset unspecified) lab58.1% ✎▃······▃·▅·█ beats 6/6hard · avg 43%▇▇▇▇▇▇ 14%sharp · at its level
Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations)0.5021 ✎▁▄▃▃▇▄█▃▅▁▁▁ beats 17/30avg 0.423▇▇▇▇▇ 13%sharp
BIG-Bench Hard lab89.8% ✎▄·····▄·▄·██ beats 6/6easy · avg 78%▇▇▇▇ 10%sharp
MMLU lab85.4% ✎▄···▄··█··▄█ beats 5/6easy · avg 77%▇▇ 5.4%typical
AGIEval English lab88.2% ✎▄····▄·▄▄··█ beats 4/5easy · avg 76%▇▇ 4.1%typical
ZebraLogic lab76.0% ✎█▄····▄·▄·▄▄ beats 5/6medium · avg 53%▇ 2.9%at its level
PopQA lab31.9% ✎▄··█···▄···█ beats 5/5hard · avg 22%▇ 2.7%blunt · at its level
IFBench (aggregation unspecified) lab47.6% ✎█·██·██····█ beats 4/5medium · avg 36%▇ 2.1%at its level

$ where to run it

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Olmo 3 32B Think · needs ~19.5 GB · from huggingface.co/lmstudio-community
    chat in the terminal
    llama cli -hf lmstudio-community/Olmo-3-32B-Think-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf lmstudio-community/Olmo-3-32B-Think-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Olmo 3 32B Think · needs ~19.5 GB · from huggingface.co/lmstudio-community

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/lmstudio-community/Olmo-3-32B-Think-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Olmo 3 32B Think · needs ~19.82 GB · from ollama.com
    chat in the terminal
    ollama run olmo-3:32b-think-q4_K_M

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Olmo 3 32B Think · needs ~18.1 GB, Mac only · from huggingface.co/lmstudio-community
    chat in the terminal
    mlx_lm.chat --model lmstudio-community/Olmo-3-32B-Think-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model lmstudio-community/Olmo-3-32B-Think-MLX-4bit

Weights and documentation: huggingface.co/allenai ↗

$ how the score was worked out · Olmo 3 75%, Medmarks: A Comprehensive Ope… 13%, LMArena leaderboard dataset 8% and 1 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Olmo 3 · lab-reported14▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 75%CC BY 4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models1▇▇▇▇ 13%CC BY 4.0
LMArena leaderboard dataset1▇▇ 8%CC BY 4.0
allenai/Olmo-3.1-32B-Think model card · lab-reported2▇ 4%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 57.9

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json