whatmodel.sh / Phi-4-mini 3.8B

Phi-4-mini 3.8B

+ contribute

Phi-4-mini 3.8B is an open-weights language model from Microsoft, released February 2025. Its published size is 3.84B parameters and a 128k-token context window. Licence: mit.

43.7±0.7#470 of 719 LLMsfrom 40 benchmark results across 10 sources

$ how it compares

The LLMs ranked either side of Phi-4-mini 3.8B, out of 828 with a score, then more from Microsoft.

#	model	score	where	RAM	released
557	Code Tulu 2 34B Allen Institute for AI	44±2	local	~72GB	2023-11
558	Qwen3 1.7B Alibaba	44±1	local	~2.6GB	2025-04
559	Phi-4-mini 3.8B Microsoft	44±1	api $0.13/M +local	~3.8GB	2025-02
560	Ministral 8B (2410) Mistral	44±1	local	~6.2GB	2024-10
561	Granite 3.2 8B Instruct IBM	44±1	local	~6.6GB	2025-02
	more from Microsoft
252	Phi-4-reasoning-plus 14B Microsoft	59±1	local	~10GB	2025-04
261	Phi-4-reasoning 14B Microsoft	58±1	api $0.22/M +local	~10GB	2025-04
365	Phi-4 14B Microsoft	53±1	api $0.087/M +local	~10GB	2024-12

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▃▃▆█▆▅▅▅▄▃▁ #294 of 300 · 2.3% of its score · 3 tests

Beats QwQ-32B-Preview, DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) here, though they rank higher overall. Trails Granite 4.0 350M, Qwen2.5 0.5B Instruct, Granite 4.0 H 350M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BFCL v3	28.2% ✎	▁▁▃▁▁▃▅█▅▄▁▁ beats 6/114	medium · avg 52%	▇ 0.9%	blunt · at its level
BFCL V4 (2025-11-03 archive, models absent from the December archive)	21.7% ✎	▃··▅▃▃·▅·▃·█ beats 0/10	hard · avg 34%	▇ 0.8%	at its level
BFCL (version not stated) lab	70.3% ✎	▄▆▄▃·█▆·▄··▂ beats 7/17	easy · avg 77%	▇ 0.6%	blunt · at its level · small run

coding▁▁▁▃▆██▇▆▄▃▁ #309 of 458 · 8.0% of its score · 4 tests

Beats Solar Open 100B (102B-A12B), GLM-4.7-Flash, Qwen3 4B Instruct 2507 here, though they rank higher overall. Trails DeepSeek LLM 67B Chat, WizardLM 70B V1.0 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
HumanEval lab	74.4% ✎	▁▁▁▁▁▃▅▅█▃▂▁ beats 140/190	easy · avg 84%	▇ 2.9%	sharp · at its level · small run
HumanEval+ lab	68.3% ✎	▁·▁▁▂▃▄▄█▅▄▃ beats 74/145	easy · avg 80%	▇ 2.6%	sharp · at its level · small run
MBPP lab	65.3% ✎	▁▁▁▄▅█▄▅▃▁▂▁ beats 57/81	easy · avg 76%	▇ 1.4%	at its level · small run
MBPP+ lab	63.8% ✎	▁··▁▁▃▄█▆▄▁▁ beats 58/120	easy · avg 71%	▇ 1.0%	at its level · small run

chat & writing█··········█ #1 of 2 * · 0.6% of its score · 1 test

test	result	vs the others in its run	difficulty	counts	why
IFEval (aggregation unspecified) lab	70.1% ✎	▁▁▃▁▃▄▄▆█▃▂▁ beats 32/141	easy · avg 75%	▇ 0.6%	at its level · small run

math▁▂▃▅▆▅▄█▄▁▁▁ #279 of 398 · 17% of its score · 5 tests

Beats Molmo 2 8B, DeepSeek-Coder-V2 Instruct (0724), Llama 3.1 Nemotron 70B Instruct here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Granite 8B Code Base (4K), Mixtral 8x7B v0.1 (base) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
GSM8K lab	88.6% ✎	█·█▄▄█···█·▄ beats 8/10	very easy · avg 91%	▇▇▇ 6.7%	sharp
MGSM lab	63.9% ✎	▅▃▃█·▅···▃·▃ beats 7/10	easy · avg 71%	▇▇ 4.9%	at its level
MATH lab	64.0% ✎	█▄██···▄▄▄·▄ beats 9/10	medium · avg 64%	▇ 3.1%	at its level
AIME 2024 lab	10.0% ✎	▁▁▁▃▁▃▁▅███▃ beats 9/85	medium · avg 45%	▇ 1.7%	sharp · at its level · noisy
MATH-500 lab	71.8% ✎	▁▃▂▃▅▇▅▇█▅▂▁ beats 14/84	easy · avg 84%	▇ 1.1%	noisy

other tests27 not in a task · 71% of its score

test	result	vs the others in its run	difficulty	counts	why
ARC Challenge lab	83.7% ✎	▂▂▂█▂···▄··▂ beats 6/10	very easy · avg 91%	▇▇▇▇ 9.5%	sharp
MMLU lab	67.3% ✎	█▄█▄▄▄·▄▄··▄ beats 6/10	easy · avg 77%	▇▇ 6.2%	at its level
CLIcK Korean (Daekeun Kim)	42.8% ✎	▆▄▄▄█▄▂·▄█·▄ beats 0/23	medium · avg 58%	▇▇ 6.1%	at its level
BIG-Bench Hard lab	70.4% ✎	██▄·█▄·▄▄··▄ beats 8/10	easy · avg 74%	▇▇ 5.7%	sharp · at its level
HAE-RAE 1.0 (Daekeun Kim)	44.5% ✎	▄▄▄▂▂█▂▄▂█▂▄ beats 1/22	medium · avg 63%	▇▇ 5.5%	at its level
OpenBookQA lab	79.2% ✎	▂··▄█▂·▂···▄ beats 1/10	easy · avg 87%	▇▇ 5.5%	at its level
MMLU-Pro lab	52.8% ✎	█▄▄·█▄▄▄·▄·▄ beats 8/10	medium · avg 57%	▇▇ 3.8%	at its level
Multilingual MMLU (Microsoft) lab	49.3% ✎	█▄▄██··█···▄ beats 2/10	easy · avg 65%	▇ 3.5%	at its level

19 more tests

test	result	vs the others in its run	difficulty	counts	why
CPTUB average (SpeakLeash, 0-5 judge scale)	43.4% ✎	▁▂▃▂▃█▃▆▅▄▂▃ beats 11/95	easy · avg 66%	▇ 3.1%	at its level
GPQA Diamond lab	36.9% ✎	▁▁▁▁▂▄▅▅███▄ beats 38/310	hard · avg 49%	▇ 2.8%	sharp
Polish EQ-Bench v2 (SpeakLeash)	50.8% ✎	▁··▁▁▂▂▆▅▆█▆ beats 25/70	medium · avg 64%	▇ 2.4%	at its level
WinoGrande lab	67.0% ✎	▃····▃▃█▃·▅▅ beats 6/10	medium · avg 74%	▇ 2.3%	at its level
HellaSwag lab	69.1% ✎	▃▅█·▃▃▃···▃▃ beats 0/10	easy · avg 84%	▇ 2.1%	at its level
BoolQ lab	81.2% ✎	▃··▃··██▃·▃▃ beats 7/10	easy · avg 85%	▇ 2.0%	at its level
Social IQA lab	72.5% ✎	▃·▃▅▅█·▃···▃ beats 2/10	easy · avg 78%	▇ 1.9%	blunt · at its level
BRIDGE clinical text, zero-shot average (87 tasks)	24.5% ✎	▁▁▁▁▁▂▃▃▄▆█▃ beats 17/97	hard · avg 32%	▇ 1.8%	blunt · at its level
KCL-Essay vanilla	6.0% ✎	▃···▁▃▂█▂▃▃▂ beats 0/23	hard · avg 18%	▇ 1.3%	typical
GPQA (subset unspecified) lab	25.2% ✎	█··▄▂··▂▂··▄ beats 3/10	very hard · avg 33%	▇ 1.0%	sharp · too hard for it · noisy
Open PT-LLM Leaderboard average (9 Portuguese tasks)	66.5% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 127/276	medium · avg 73%	▇ 0.9%	at its level · noisy
BigCodeBench (Complete) lab	43.0% ✎	▂▁▁▃▂▅▇█▅▂▂▂ beats 35/60	medium · avg 51%	▇ 0.9%	at its level · small run
TruthfulQA MC2 lab	66.4% ✎	▂·█▂▃▄····▂▂ beats 6/10	easy · avg 69%	▇ 0.6%	blunt · at its level
PIQA lab	77.6% ✎	▃·▅·▃█·▃▅··▃ beats 4/10	easy · avg 83%	▇ 0.6%	blunt · at its level
BigCodeBench (Instruct) lab	33.8% ✎	▁▁··▁▁▂▅█▅▅▅ beats 20/49	medium · avg 36%	▇ 0.6%	at its level · small run
KMMLU-HARD (Daekeun Kim, zero-shot)	25.3% ✎	█···▁▁▁▂▃·▆▂ beats 5/23	very hard · avg 29%	▇ 0.5%	sharp · too hard for it
KCL-MCQA vanilla	22.0% ✎	▃···█▂▆▆██▆▂ beats 0/23	very hard · avg 30%	▇ 0.5%	too hard for it
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	96.6% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 96/140	very easy · avg 94%	▇ 0.3%	blunt · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	76.5% ✎	▃▂▁▂▅█▂▃▄▃▁▁ beats 1/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · Microsoft Azure · $0.13/M · $0.075 in / $0.3 out per 1M tokens · 128k context · azure.microsoft.com ↗ · price sourceUSD per 1M tokens, Azure AI Foundry Models pay-as-you-go (Microsoft models table), 'Phi-4-mini' (Azure model id Phi-4-mini-instruct, GA), 128K context: input $0.075, output $0.30, region US East. Fine-tuned hosting/training rates excluded. Currency: USD. Microsoft Azure official pricing page (price cells carry the amounts in the page's own data), observed 2026-09-25.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Phi-4-mini 3.8B · needs ~2.5 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Phi-4-mini-instruct-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Phi-4-mini-instruct-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Phi-4-mini 3.8B · needs ~2.5 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Phi-4-mini-instruct-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Phi-4-mini 3.8B · needs ~2.8 GB · from ollama.com
chat in the terminal
ollama run phi4-mini:3.8b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Phi-4-mini 3.8B · needs ~2.2 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Phi-4-mini-instruct-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Phi-4-mini-instruct-4bit

Official weights and documentation: huggingface.co/microsoft ↗

$ how the score was worked out · microsoft/Phi-4-mini-instruct… 59%, Microsoft — Phi-4-Mini Techni… 16%, Daekeun Kim Korean LLM benchm… 12% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
microsoft/Phi-4-mini-instruct model card · lab-reported	16	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 59%	MIT
Microsoft — Phi-4-Mini Technical Report · lab-reported	11	▇▇▇▇▇ 16%	CC BY 4.0
Daekeun Kim Korean LLM benchmark runs	3	▇▇▇▇ 12%	MIT
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 3%	MIT
SpeakLeash Polish EQ-Bench leaderboard	1	▇ 2%	Apache-2.0
Korean Canonical Legal Benchmark: Toward Knowledge-Independent Evaluation of LLMs’ Legal Reasoning Capabilities	2	▇ 2%	CC BY 4.0
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)	1	▇ 2%	MIT
Berkeley Function Calling Leaderboard result archive	2	▇ 2%	Apache-2.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	2	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 43.7

$ quantization evidence · 2 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
FP8 W8A8 dynamic (weights symmetric static per-channel, activations symmetric dynamic per-token)	MMLU with chain-of-thought prompting (lm-evaluation-harness, vLLM)	0.67 → 0.67	99.9%	source ↗
FP8 W8A8 dynamic (weights symmetric static per-channel, activations symmetric dynamic per-token)	MMLU-Pro (lm-evaluation-harness, vLLM)	0.468 → 0.456	97.5%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json