whatmodel.sh / Phi-3.5-mini-instruct

Phi-3.5-mini-instruct

+ contribute

Phi-3.5-mini-instruct is an open-weights language model from Microsoft, released August 2024. Its published size is 3.82B parameters and a 128k-token context window. Licence: mit.

Catalogue identity notes

Maker repo microsoft/Phi-3.5-mini-instruct @2fe1924: licence tag mit. 3,821,079,552 parameters from the Hugging Face safetensors metadata. Dense, so active = total. Released 2024-08: month the maker's Hugging Face repo was created (createdAt 2024-08-16). Ollama tags phi3.5:3.8b-mini-instruct-q4_K_M, phi3.5:3.8b-mini-instruct-q8_0 are the library's instruct/chat builds of this checkpoint (tag names on the pinned tags page); download sizes are estimated from the parameter count.

43.3±0.6#477 of 719 LLMsfrom 38 benchmark results across 10 sources

$ how it compares

The LLMs ranked either side of Phi-3.5-mini-instruct, out of 828 with a score, then more from Microsoft.

#	model	score	where	RAM	released
564	LFM2.5 1.2B Instruct Liquid AI	44±2	local	~2.2GB	2026-01
565	Granite 3.0 8B Instruct IBM	43±1	local	~6.6GB	2024-10
566	Phi-3.5-mini-instruct Microsoft	43±1	local	~3.7GB	2024-08
567	Hermes 2 Theta Llama 3 8B Nous Research	43±2	local	~6.6GB	2024-05
568	Yi 34B Chat 01.AI	43±1	local	~23GB	2023-11
	more from Microsoft
252	Phi-4-reasoning-plus 14B Microsoft	59±1	local	~10GB	2025-04
261	Phi-4-reasoning 14B Microsoft	58±1	api $0.22/M +local	~10GB	2025-04
365	Phi-4 14B Microsoft	53±1	api $0.087/M +local	~10GB	2024-12

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #227 of 300 * · 0.4% of its score · 1 test

Beats o3-mini, Nova 2 Lite, Ministral 3 14B Reasoning 2512 here, though they rank higher overall. Trails SmolLM3 3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BFCL (version not stated) lab	66.1% ✎	▄▆▄▂·█▆·▄··▂ beats 5/17	easy · avg 77%	▇ 0.4%	blunt · at its level · small run

coding▁▁▁▃▆██▇▆▄▃▁ #330 of 458 · 13% of its score · 5 tests

Beats GLM-4.7-Flash, Qwen3-VL 4B Instruct, LFM2.5 2.6B here, though they rank higher overall. Trails Phi-3-mini-128k-instruct (Apr 2024), DeepSeek LLM 67B Chat, WizardLM 70B V1.0 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CRUXEval (ZeroEval, 800 problems)	42.1% ✎	▁▂▆▃▆█▆▃▂·▁▁ beats 6/47	medium · avg 61%	▇▇▇ 7.7%	sharp · at its level
HumanEval lab	70.1% ✎	▁▁▁▁▁▃▅▅█▃▂▁ beats 119/190	easy · avg 84%	▇ 1.9%	sharp · at its level · small run
HumanEval+ lab	62.8% ✎	▁·▁▁▂▃▄▄█▅▄▃ beats 62/145	easy · avg 80%	▇ 1.7%	sharp · at its level · small run
MBPP lab	70.0% ✎	▁▁▁▄▅█▄▅▃▁▂▁ beats 65/81	easy · avg 76%	▇ 0.9%	at its level · small run
MBPP+ lab	63.8% ✎	▁··▁▁▃▄█▆▄▁▁ beats 57/120	easy · avg 71%	▇ 0.7%	at its level · small run

chat & writing█··········█ #2 of 2 * · 0.4% of its score · 1 test

test	result	vs the others in its run	difficulty	counts	why
IFEval (aggregation unspecified) lab	50.6% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 9/141	easy · avg 75%	▇ 0.4%	at its level · small run

math▁▂▃▅▆▅▄█▄▁▁▁ #334 of 398 · 18% of its score · 5 tests

Beats GPT-3.5 Turbo (0125), Qwen2-VL 7B Instruct, DBRX Instruct here, though they rank higher overall. Trails Qwen3 0.6B, Gemma 7B (base), LFM2 700M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
GSM8K (ZeroEval, zero-shot)	82.0% ✎	▁▄·▁▄█▅▅▅███ beats 13/43	very easy · avg 89%	▇▇ 5.7%	sharp
GSM8K lab	76.9% ✎	█·█▄▄█···█·▄ beats 1/10	very easy · avg 91%	▇▇ 4.5%	sharp
MGSM lab	49.6% ✎	▅▃▃█·▅···▃·▃ beats 2/10	easy · avg 71%	▇ 3.2%	at its level
MATH Level 5 (ZeroEval, 721 problems)	18.7% ✎	▁▁▁▆█▂▇▇▄··▂ beats 6/38	hard · avg 34%	▇ 2.3%	sharp
MATH lab	49.8% ✎	█▄██···▄▄▄·▄ beats 5/10	medium · avg 64%	▇ 2.0%	at its level

other tests26 not in a task · 68% of its score

test	result	vs the others in its run	difficulty	counts	why
MMLU-Redux (ZeroEval, 2,778 questions)	67.7% ✎	▁·▁▁▄▄▄▅█▄▂▁ beats 12/53	easy · avg 77%	▇▇▇▇▇▇ 14%	at its level
LiveBench 2024-07-26	28.3% ✎	▁▁▁▂▂▃▂█▆▃▅▃ beats 23/61	medium · avg 41%	▇▇▇▇ 10%	at its level
ARC Challenge lab	84.6% ✎	▂▂▂█▃···▄··▂ beats 7/10	very easy · avg 91%	▇▇▇ 6.4%	sharp
MMLU lab	65.5% ✎	█▄█▄▄▄·▄▄··▄ beats 5/10	easy · avg 77%	▇▇ 4.1%	at its level
CLIcK Korean (Daekeun Kim)	43.0% ✎	▆▄▄▄█▄▂·▄█·▄ beats 1/23	medium · avg 58%	▇▇ 4.0%	at its level
BIG-Bench Hard lab	63.1% ✎	██▄·█▄·▄▄··▄ beats 5/10	easy · avg 74%	▇▇ 3.8%	sharp · at its level
OpenBookQA lab	81.2% ✎	▂··▄█▂·▂···▄ beats 6/10	easy · avg 87%	▇ 3.7%	at its level
HAE-RAE 1.0 (Daekeun Kim)	44.2% ✎	▄▄▄▂▂█▂▄▂█▂▄ beats 0/22	medium · avg 63%	▇ 3.6%	at its level

18 more tests

test	result	vs the others in its run	difficulty	counts	why
MMLU-Pro lab	47.4% ✎	█▄▄·█▄▄▄·▄·▄ beats 6/10	medium · avg 57%	▇ 2.4%	at its level
Multilingual MMLU (Microsoft) lab	51.8% ✎	█▄▄██··█···▄ beats 3/10	easy · avg 65%	▇ 2.3%	at its level
CPTUB average (SpeakLeash, 0-5 judge scale)	40.2% ✎	▁▂▃▂▃█▃▆▅▄▂▃ beats 9/95	easy · avg 66%	▇ 2.0%	at its level
ZebraLogic (ZeroEval, 1,000 puzzles)	6.4% ✎	▁▁▃▆██▁▁·▁▁▁ beats 3/57	hard · avg 18%	▇ 2.0%	sharp · too hard for it
WinoGrande lab	72.2% ✎	▃····▃▃█▃·▅▅ beats 8/10	medium · avg 74%	▇ 1.5%	at its level
HellaSwag lab	72.2% ✎	▃▅█·▃▃▃···▃▃ beats 1/10	easy · avg 84%	▇ 1.4%	at its level
BoolQ lab	77.7% ✎	▃··▃··██▃·▃▃ beats 2/10	easy · avg 85%	▇ 1.3%	at its level
Social IQA lab	75.1% ✎	▃·▃▅▅█·▃···▃ beats 6/10	easy · avg 78%	▇ 1.2%	blunt · at its level
BRIDGE clinical text, zero-shot average (87 tasks)	25.4% ✎	▁▁▁▁▁▂▃▃▄▆█▃ beats 20/97	hard · avg 32%	▇ 1.2%	blunt · at its level
GPQA (subset unspecified) lab	26.6% ✎	█··▄▃··▂▂··▄ beats 6/10	very hard · avg 33%	▇ 0.6%	sharp · too hard for it · noisy
Open PT-LLM Leaderboard average (9 Portuguese tasks)	66.8% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 131/276	medium · avg 73%	▇ 0.6%	at its level · noisy
BigCodeBench (Complete) lab	40.4% ✎	▂▁▁▃▂▅▇█▅▂▂▂ beats 29/60	medium · avg 51%	▇ 0.6%	at its level · small run
TruthfulQA MC2 lab	65.2% ✎	▂·█▃▂▄····▂▂ beats 5/10	easy · avg 69%	▇ 0.4%	blunt · at its level
PIQA lab	78.2% ✎	▃·▅·▃█·▃▅··▃ beats 5/10	easy · avg 83%	▇ 0.4%	blunt · at its level
BigCodeBench (Instruct) lab	14.3% ✎	▁▃··▁▁▂▅█▅▅▅ beats 1/49	medium · avg 36%	▇ 0.4%	at its level · small run
KMMLU-HARD (Daekeun Kim, zero-shot)	24.0% ✎	█···▁▁▁▂▃·▆▂ beats 4/23	very hard · avg 29%	▇ 0.3%	sharp · too hard for it
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	95.9% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 85/140	very easy · avg 94%	▇ 0.2%	blunt · noisy
LIBRA Mini (Russian long context, May 2026 version)	28.3% ✎	▄···▄█·██··▄ beats 0/8	medium · avg 38%	▇ 0.1%	blunt · at its level

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Phi-3.5-mini-instruct · needs ~2.4 GB · from huggingface.co/bartowski
chat in the terminal
llama cli -hf bartowski/Phi-3.5-mini-instruct-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf bartowski/Phi-3.5-mini-instruct-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Phi-3.5-mini-instruct · needs ~2.4 GB · from huggingface.co/bartowski
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Phi-3.5-mini-instruct · needs ~2.79 GB · from ollama.com
chat in the terminal
ollama run phi3.5:3.8b-mini-instruct-q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Phi-3.5-mini-instruct · needs ~2.1 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Phi-3.5-mini-instruct-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Phi-3.5-mini-instruct-4bit

Official weights and documentation: huggingface.co/microsoft ↗

$ how the score was worked out · microsoft/Phi-4-mini-instruct… 39%, WildEval ZeroEval published r… 32%, LiveBench model judgments (HF… 10% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
microsoft/Phi-4-mini-instruct model card · lab-reported	16	▇▇▇▇▇▇▇▇▇▇▇▇ 39%	MIT
WildEval ZeroEval published results	5	▇▇▇▇▇▇▇▇▇▇ 32%	Apache-2.0
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets	1	▇▇▇ 10%	Apache-2.0 (stated in the LiveBench datasheet)
Daekeun Kim Korean LLM benchmark runs	3	▇▇ 8%	MIT
Microsoft — Phi-4-Mini Technical Report · lab-reported	8	▇▇ 7%	CC BY 4.0
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 2%	MIT
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)	1	▇ 1%	MIT
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
ai-forever LIBRA leaderboard (Russian long-context benchmark)	1	▇ 0%	MIT
prior · pull toward the middle		▇ 0%
weighted average		= 43.3

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json