whatmodel.sh / Granite 4.0 H Small (32B-A9B)

Granite 4.0 H Small (32B-A9B)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Granite 4.0 H Small (32B-A9B) is an open-weights language model from IBM, released September 2025. Its published size is 32.2B parameters (9B active per token) and a 131k-token context window. Licence: apache-2.0.

51.8±0.7#314 of 719 LLMsfrom 39 benchmark results across 7 sources

$ how it compares

The LLMs ranked either side of Granite 4.0 H Small (32B-A9B), out of 828 with a score, then more from IBM.

#modelscorewhereRAMreleased
378ERNIE 4.5 21B-A3B Baidu52±1comparison only2025-06
379Bielik 11B v2.2 Instruct SpeakLeash52±3local~8.5GB2024-08
380Granite 4.0 H Small (32B-A9B) IBM52±1api $0.11/M +local~21GB2025-09
381Llama 3.1 Nemotron 70B Instruct NVIDIA52±1local~43GB2024-10
382Palmyra X4 (palmyra-x-004) Writer52±4comparison only2024-10
more from IBM
192Granite 4.2 30B IBM63±1api $0.28/M +local~19GB2026-08
239Granite 4.2 8B IBM60±1api $0.11/M +local~6.6GB2026-08
276Granite 4.2 3B IBM57±1api $0.052/M +local~3.7GB2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▄·▄········█ #1 of 4 * · 0.7% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
BFCL v3 lab64.7% ✎▁▁▁▁▁▃▅█▆▅▄▁ beats 123/170medium · avg 60%▇ 0.7%blunt · at its level
coding█···▄······▄ #1 of 4 · 16% of its score · 7 tests
testresultvs the others in its rundifficultycountswhy
HumanEval lab88.0% ✎▁▁▁▁▁▃▅▅█▃▃▁ beats 180/190very easy · avg 86%▇▇ 4.1%sharp
HumanEval+ lab83.0% ✎▁·▁▁▂▃▄▄█▅▄▃ beats 114/145easy · avg 81%▇▇ 3.9%sharp
MBPP lab84.0% ✎▁▁▁▄▅█▄▅▃▁▃▁ beats 77/81easy · avg 79%▇ 1.9%typical
BigCodeBench lab46.2% ✎▅····▃·█▃▅█▃ beats 12/12medium · avg 42%▇ 1.9%at its level
MultiPL-E lab57.4% ✎▃··▄·▅▅▇▄█▃▄ beats 12/31medium · avg 58%▇ 1.7%at its level
MBPP+ lab71.0% ✎▁··▁▁▃▄█▆▄▁▁ beats 87/120easy · avg 70%▇ 1.4%at its level
CRUXEval-O lab50.2% ✎▂▂██▅▃▁·▁·▁▁ beats 45/51medium · avg 48%▇ 1.0%blunt · at its level
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #231 of 393 · 6.5% of its score · 3 tests

Beats Mistral Small 4 (119B-A6.5B), Ministral 3 14B Reasoning 2512, Qwen2.5 Coder 32B Instruct here, though they rank higher overall. Trails Flan-T5 XXL, LFM2.5 1.2B Instruct, Gemma 3 4B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1287.0 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 117/379avg 1268▇▇ 4.2%blunt
IFEval instruction-level strict lab89.9% ✎▄▄····▄▄█▄·█ beats 7/8very easy · avg 88%▇ 1.2%typical
IFEval prompt-level strict lab85.2% ✎▁▁··▂··▂▇█▃▃ beats 14/26easy · avg 81%▇ 1.2%typical
math▁▂▃▅▆▅▄█▄▁▁▁ #224 of 398 · 22% of its score · 8 tests

Beats GPT-4o (2024-08-06), Command A (03-2025), GPT-4o (2024-11-20) here, though they rank higher overall. Trails Olmo 3 7B Instruct, DeepSeek R1 Distill Llama 8B, Phi-4-mini-reasoning 3.8B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K +183.9% ✎█····█··▄·▄█ beats 2/7very easy · avg 86%▇▇▇ 7.0%sharp · several runs
DeepMind Mathematics lab59.3% ✎▅····▅·█▃▃·▅ beats 7/10medium · avg 55%▇ 3.0%at its level
Minerva MATH lab74.0% ✎▄·▂▂·██▆·▆▄▄ beats 19/21easy · avg 70%▇ 2.6%at its level
MATH (subset unspecified)75.7% ✎█▄··▄█▄····▄ beats 2/7easy · avg 81%▇ 2.5%typical
GSM-Symbolic lab87.4% ✎▃▃····▃·▅▅█▃ beats 10/10easy · avg 83%▇ 2.1%typical
PolyMath (Marco evaluation)28.9% ✎▄··▄██··▄··▄ beats 4/7hard · avg 30%▇ 2.0%at its level
MGSM (Marco evaluation)75.7% ✎▄·▄·▄▄▄·▄··█ beats 5/7easy · avg 70%▇ 1.7%at its level
MGSM, 5 languages (en, es, fr, ja, zh) lab38.7% ✎█·▄▄██···█·▄ beats 5/10medium · avg 46%▇ 1.0%at its level
other tests17 not in a task · 55% of its score
testresultvs the others in its rundifficultycountswhy
MMLU +176.7% ✎▄···▄▄·▄█··█ beats 5/7easy · avg 75%▇▇▇ 7.8%at its level · several runs
GPQA Diamond38.6% ✎▄···█·▄·█·▄▄ beats 4/7hard · avg 38%▇▇▇ 7.2%sharp
MMLU-Pro +157.1% ✎▄····█·▄█·▄▄ beats 4/7medium · avg 56%▇▇▇ 6.5%at its level · several runs
Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations)0.4452 ✎▁▄▃▃▇▄█▃▅▁▁▁ beats 10/30avg 0.423▇▇▇ 6.4%sharp
GPQA (subset unspecified) lab40.6% ✎▂▁▂▂▄▃▆▃▅█▄▃ beats 40/104hard · avg 37%▇▇ 5.0%sharp
BIG-Bench Hard lab81.6% ✎▁▁▁▁▂▄▅▅█▃▄▁ beats 111/129easy · avg 76%▇ 3.7%sharp · at its level
Medmarks-V weighted mean win rate (61 models, 71 configurations)44.5% ✎▂▃▅▂▆▃█▅█▅▃▆ beats 10/34medium · avg 42%▇ 3.1%blunt · at its level
MMLU-ProX-Lite (Marco)51.6% ✎█··▄··█··█·▄ beats 6/7medium · avg 43%▇ 3.0%at its level
9 more tests
testresultvs the others in its rundifficultycountswhy
INCLUDE lab64.0% ✎▂····▂▂▄▄▄█▄ beats 14/36medium · avg 58%▇ 2.6%at its level
GlobalMMLU (Marco evaluation)67.4% ✎▃··▃▃··▅···█ beats 5/7medium · avg 57%▇ 2.2%at its level
MMMLU lab69.7% ✎▂▁▁▂▃▂▄▄▅██▃ beats 19/46medium · avg 63%▇ 2.1%at its level
MMMLU (Marco evaluation)68.1% ✎▄···▄·█·▄·▄█ beats 5/7medium · avg 60%▇ 1.9%at its level
AGIEval (Marco suite unspecified)44.7% ✎▄··▄█▄··▄··█ beats 2/7medium · avg 47%▇ 1.5%blunt · at its level
DiagnosticIQ industrial maintenance MCQ (macro accuracy, 4 options) lab53.2% ✎▁··▃▁▃▁▅█▅▁▅ beats 4/24medium · avg 52%▇ 0.8%blunt · at its level
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy84.8% ✎▃·▁▁▂▁▂▅█▇▄▁ beats 83/100medium · avg 68%▇ 0.7%at its level · noisy
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy86.5% ✎▁▁▁▁▁▁▁▂▄█▄▃ beats 103/146easy · avg 83%▇ 0.4%noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)94.8% ✎▁▂▁▂▅█▂▃▄▃▁▁ beats 88/98very easy · avg 90%▇ 0.0%blunt

$ where to run it

Through an API

api · Replicate · $0.11/M · $0.06 in / $0.25 out per 1M tokens · 131k context · replicate.com ↗ · price sourceUSD per 1M tokens, Replicate official model ibm-granite/granite-4.0-h-small, billed per input/output token (billing table on the model page; A100 80GB backing hardware not billed separately for this official model). Currency: USD. Replicate official model page, observed 2026-09-25.

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Granite 4.0 H Small (32B-A9B) · needs ~19.5 GB · from huggingface.co/ibm-granite
    chat in the terminal
    llama cli -hf ibm-granite/granite-4.0-h-small-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf ibm-granite/granite-4.0-h-small-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Granite 4.0 H Small (32B-A9B) · needs ~19.5 GB · from huggingface.co/ibm-granite

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/ibm-granite/granite-4.0-h-small-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Granite 4.0 H Small (32B-A9B) · needs ~19.82 GB · from ollama.com
    chat in the terminal
    ollama run granite4:small-h

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Granite 4.0 H Small (32B-A9B) · needs ~18.1 GB, Mac only · from huggingface.co/lmstudio-community
    chat in the terminal
    mlx_lm.chat --model lmstudio-community/granite-4.0-h-small-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model lmstudio-community/granite-4.0-h-small-MLX-4bit

Weights and documentation: huggingface.co/ibm-granite ↗

$ how the score was worked out · ibm-granite/granite-4.0-h-sma… 50%, Marco-MoE: Open Multilingual… 34%, Medmarks: A Comprehensive Ope… 10% and 4 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
ibm-granite/granite-4.0-h-small model card · lab-reported21▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 50%Apache-2.0
Marco-MoE: Open Multilingual Mixture-of-Expert Language Models with Efficient Upcycling11▇▇▇▇▇▇▇▇▇▇ 34%CC BY 4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models2▇▇▇ 10%CC BY 4.0
LMArena leaderboard dataset1▇ 4%CC BY 4.0
EuroEval leaderboards (results collection), English generative leaderboard2▇ 1%MIT
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1) · lab-reported1▇ 1%CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)1▇ 0%Apache-2.0
prior · pull toward the middle▇ 0%
weighted average= 51.8
$ quantization evidence · 18 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precisiontestfull → quantizedkept
FP8 (IBM official ibm-granite/granite-4.0-h-small-FP8 checkpoint; recipe not described in this card)GSM8K 5-shot strict-match (lm-evaluation-harness, vLLM)0.852 → 0.853100.1%source ↗
FP8 (IBM official ibm-granite/granite-4.0-h-small-FP8 checkpoint; recipe not described in this card)MMLU 5-shot acc (lm-evaluation-harness, vLLM)0.771 → 0.772100.0%source ↗
FP8 (IBM official ibm-granite/granite-4.0-h-small-FP8 checkpoint; recipe not described in this card)OpenLLM v1 average of 6 tasks (ARC-Challenge 25-shot, GSM8K 5-shot, HellaSwag 10-shot, MMLU 5-shot, TruthfulQA MC2 0-shot, Winogrande 5-shot; lm-evaluation-harness `openllm`, vLLM)0.766 → 0.766100.0%source ↗
FP8 (IBM official ibm-granite/granite-4.0-h-small-FP8 checkpoint; recipe not described in this card)IFEval 0-shot instruction-level strict acc (OpenLLM v2 config)0.875 → 0.87299.6%source ↗
FP8 (IBM official ibm-granite/granite-4.0-h-small-FP8 checkpoint; recipe not described in this card)MMLU-Pro 5-shot (OpenLLM v2 config; value as published)0.48 → 0.47699.2%source ↗
FP8 (IBM official ibm-granite/granite-4.0-h-small-FP8 checkpoint; recipe not described in this card)OpenLLM v2 average of 6 tasks (IFEval, BBH, MATH-Hard, GPQA, MuSR, MMLU-Pro; lm-evaluation-harness `leaderboard`, chat template, vLLM; values exactly as published, per-task normalisation not stated)0.542 → 0.53698.8%source ↗
FP8 block-wise (llm-compressor FP8_BLOCK)GSM8K 5-shot strict-match (lm-evaluation-harness, vLLM)0.852 → 0.855100.4%source ↗
FP8 block-wise (llm-compressor FP8_BLOCK)MMLU 5-shot acc (lm-evaluation-harness, vLLM)0.771 → 0.772100.1%source ↗
FP8 block-wise (llm-compressor FP8_BLOCK)OpenLLM v1 average of 6 tasks (ARC-Challenge 25-shot, GSM8K 5-shot, HellaSwag 10-shot, MMLU 5-shot, TruthfulQA MC2 0-shot, Winogrande 5-shot; lm-evaluation-harness `openllm`, vLLM)0.766 → 0.766100.0%source ↗
FP8 block-wise (llm-compressor FP8_BLOCK)MMLU-Pro 5-shot (OpenLLM v2 config; value as published)0.48 → 0.47999.9%source ↗
FP8 block-wise (llm-compressor FP8_BLOCK)IFEval 0-shot instruction-level strict acc (OpenLLM v2 config)0.875 → 0.86799.0%source ↗
FP8 block-wise (llm-compressor FP8_BLOCK)OpenLLM v2 average of 6 tasks (IFEval, BBH, MATH-Hard, GPQA, MuSR, MMLU-Pro; lm-evaluation-harness `leaderboard`, chat template, vLLM; values exactly as published, per-task normalisation not stated)0.542 → 0.53398.4%source ↗
FP8 dynamic (llm-compressor FP8_DYNAMIC)MMLU 5-shot acc (lm-evaluation-harness, vLLM)0.771 → 0.772100.0%source ↗
FP8 dynamic (llm-compressor FP8_DYNAMIC)IFEval 0-shot instruction-level strict acc (OpenLLM v2 config)0.875 → 0.87499.9%source ↗
FP8 dynamic (llm-compressor FP8_DYNAMIC)OpenLLM v1 average of 6 tasks (ARC-Challenge 25-shot, GSM8K 5-shot, HellaSwag 10-shot, MMLU 5-shot, TruthfulQA MC2 0-shot, Winogrande 5-shot; lm-evaluation-harness `openllm`, vLLM)0.766 → 0.76599.9%source ↗
FP8 dynamic (llm-compressor FP8_DYNAMIC)GSM8K 5-shot strict-match (lm-evaluation-harness, vLLM)0.852 → 0.84899.6%source ↗
FP8 dynamic (llm-compressor FP8_DYNAMIC)MMLU-Pro 5-shot (OpenLLM v2 config; value as published)0.48 → 0.47699.1%source ↗
FP8 dynamic (llm-compressor FP8_DYNAMIC)OpenLLM v2 average of 6 tasks (IFEval, BBH, MATH-Hard, GPQA, MuSR, MMLU-Pro; lm-evaluation-harness `leaderboard`, chat template, vLLM; values exactly as published, per-task normalisation not stated)0.542 → 0.5397.8%source ↗

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json