whatmodel.sh / Phi-4-reasoning 14B
Phi-4-reasoning 14B
↗ share
+ contribute
Phi-4-reasoning 14B is an open-weights language model from Microsoft, released April 2025. Its published size is 14.7B parameters and a 33k-token context window. Licence: mit.
$ how it compares
The LLMs ranked either side of Phi-4-reasoning 14B, out of 828 with a score, then more from Microsoft.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 259 |  Olmo 3.1 32B Think | 58±1 | local | ||
| 260 |  DeepSeek R1 Distill Llama 70B | 58±1 | api $0.80/M +local | ||
| 261 | Phi-4-reasoning 14B | 58±1 | api $0.22/M +local | ||
| 262 | Olmo 3 32B Think | 58±1 | local | ||
| 263 |  Magistral Small 1.2 | 58±1 | api $0.75/M +local | ||
| more from Microsoft | |||||
| 252 | Phi-4-reasoning-plus 14B | 59±1 | local | ||
| 365 | Phi-4 14B | 53±1 | api $0.087/M +local | ||
| 422 | WizardLM-2 8x22B | 50±1 | local | ||
$ what it's good at
coding▁▁▁▃▆██▇▆▄▃▁ #143 of 458 * · 13% of its score · 3 tests
Beats Gemini 2.5 Pro, Gemini 3.1 Flash-Lite, Qwen3-Coder-Next (80B-A3B) here, though they rank higher overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| HumanEval+ lab +1 | 92.9% ✎ | ▁·▁▁▂▃▄▄█▅▄▃ beats 141/145 | 13% |
| LiveCodeBench (Aug 1, 2024–Feb 1, 2025) lab | 53.8% ✎ | █··········█ beats 1/1 | — |
| LiveCodeBench 2024-08 to 2025-01 (version not stated) lab | 53.8% ✎ | █··········█ beats 1/1 | — |
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #174 of 393 · 11% of its score · 3 tests
Beats Seed-OSS 36B Instruct, Nemotron-Cascade-14B-Thinking, Qwen3-VL 32B Instruct here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, LFM2.5 1.2B Thinking, Gemma 3 12B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena-Hard (version/judge unspecified) lab +1 | 73.3% ✎ | ▃▃▂▅▃▆▅█▅▅▂▃ beats 22/30 | 7.8% |
| IFEval strict (prompt/instruction level unspecified) lab | 83.4% ✎ | █·········██ beats 1/2 | 1.9% |
| IFEval (aggregation unspecified) lab | 83.4% ✎ | ▁▁▃▁▂▄▄▆█▃▂▁ beats 64/141 | 1.3% |
math▁▂▃▅▆▅▄█▄▁▁▁ #173 of 398 · 20% of its score · 3 tests
Beats MiniMax-M2.1, Mistral Medium 3.5 (128B), GPT-4.1 here, though they rank higher overall. Trails OpenThinker3 7B, Olmo 3 7B Think, OpenThinker2 32B here, though they rank lower overall.
other tests9 not in a task · 55% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond lab +1 | 67.1% ✎ | █··▄··▄···█▄ beats 1/6 | 20% |
| Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations) | 0.4815 ✎ | ▁▄▃▃▇▄█▃▅▁▁▁ beats 15/30 | 12% |
| MMLU-Pro lab +1 | 74.3% ✎ | ▁▁·▁▁▁▃▂▄▅█▁ beats 100/188 | 8.5% |
| Medmarks-V weighted mean win rate (61 models, 71 configurations) | 48.1% ✎ | ▂▃▅▂▆▃█▅█▅▃▆ beats 14/34 | 5.8% |
| FlenQA (3K-token subset) lab +1 | 97.7% ✎ | ▄···▄····▄·█ beats 3/4 | 5.0% |
| Open PT-LLM Leaderboard average (9 Portuguese tasks) | 67.9% ✎ | ▃▁▁▁▁▁▁▁▂█▅▁ beats 142/276 | 1.3% |
| Omni-MATH lab +1 | 76.6% ✎ | █·█·█·██··██ beats 4/6 | 1.1% |
| EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy | 48.7% ✎ | ▃·▁▁▂▁▃▅█▇▄▁ beats 25/100 | 1.0% |
1 more test
| test | result | vs the others in its run | counts |
|---|---|---|---|
| EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy | 83.5% ✎ | ▁▁▁▁▁▁▁▂▄█▄▃ beats 88/146 | 0.6% |
$ where to run it
Through an API
api · Microsoft Azure · $0.22/M · $0.125 in / $0.5 out per 1M tokens · 33k context · azure.microsoft.com ↗ · price sourceUSD per 1M tokens, Azure AI Foundry Models pay-as-you-go (Microsoft models table), 'Phi-4-reasoning' (GA; not Phi-4-reasoning-plus), 32K context: input $0.125, output $0.50, region US East. Currency: USD. Microsoft Azure official pricing page (price cells carry the amounts in the page's own data), observed 2026-09-25.
llama.cpp · recommended
version
- Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.macOS / Linux
curl -LsSf https://llama.app/install.sh | shWindowsirm https://llama.app/install.ps1 | iex - Run Phi-4-reasoning 14B · needs ~9.1 GB · from huggingface.co/lmstudio-communitychat in the terminal
llama cli -hf lmstudio-community/Phi-4-reasoning-GGUF:Q4_K_M - Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Phi-4-reasoning-GGUF:Q4_K_M
LM Studio
version
- Install LM Studio · Open the app once before using the lms command.
- Run Phi-4-reasoning 14B · needs ~9.1 GB · from huggingface.co/lmstudio-community
- Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Phi-4-reasoning-GGUF@Q4_K_M -y - Then run lms ls to see its name, and lms chat followed by that name
Ollama
version
- Install Ollama
download it (macOS / Windows) ↗
Linuxcurl -fsSL https://ollama.com/install.sh | sh - Run Phi-4-reasoning 14B · needs ~9.32 GB · from ollama.comchat in the terminal
ollama run phi4-reasoning:14b
MLX (Mac)
version
- Install MLX (Mac) · Apple silicon Macs only, text only.with uv
uv tool install mlx-lmorpip install mlx-lm - Run Phi-4-reasoning 14B · needs ~8.2 GB, Mac only · from huggingface.co/lmstudio-communitychat in the terminal
mlx_lm.chat --model lmstudio-community/Phi-4-reasoning-MLX-4bit - Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Phi-4-reasoning-MLX-4bit
Official weights and documentation: huggingface.co/microsoft ↗
$ how the score was worked out · Microsoft — Phi-4-reasoning T… 45%, microsoft/Phi-4-reasoning mod… 34%, Medmarks: A Comprehensive Ope… 18% and 2 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Microsoft — Phi-4-reasoning Technical Report · lab-reported | 10 | 45% |
 microsoft/Phi-4-reasoning model card · lab-reported | 9 | 34% |
| Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models | 2 | 18% |
 EuroEval leaderboards (results collection), English generative leaderboard | 2 | 2% |
| Open Portuguese LLM Leaderboard (archived) | 1 | 1% |
| prior · pull toward the middle | 0% | |
| weighted average | = 57.9 |
not counted (2)
| microsoft/Phi-4-reasoning model card | LiveCodeBench (Aug 1, 2024–Feb 1, 2025) [correct] | 53.8% | fewer than three models have this test yet, or this source ran it for this model alone |
| Microsoft — Phi-4-reasoning Technical Report | LiveCodeBench 2024-08 to 2025-01 (version not stated) [correct] | 53.8% | fewer than three models have this test yet, or this source ran it for this model alone |
$ quantization evidence · 4 measurements
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.
| precision | test | full → quantized | kept | |
|---|---|---|---|---|
| FP8 W8A8 dynamic (weights symmetric static per-channel, activations symmetric dynamic per-token) | AIME 2025 (lighteval, vLLM, reasoning parser) | 0.613 → 0.646 | 105.4% | source ↗ |
| FP8 W8A8 dynamic (weights symmetric static per-channel, activations symmetric dynamic per-token) | GPQA Diamond (lighteval, vLLM) | 0.646 → 0.665 | 102.9% | source ↗ |
| FP8 W8A8 dynamic (weights symmetric static per-channel, activations symmetric dynamic per-token) | MMLU-Pro 0-shot chat (lm-evaluation-harness `mmlu_pro_chat`, vLLM) | 0.765 → 0.768 | 100.5% | source ↗ |
| FP8 W8A8 dynamic (weights symmetric static per-channel, activations symmetric dynamic per-token) | MATH-500 (lighteval, vLLM) | 0.9 → 0.886 | 98.4% | source ↗ |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json