whatmodel.sh / Phi-3.5-vision-instruct

Phi-3.5-vision-instruct

+ contribute

Phi-3.5-vision-instruct is an open-weights language model from Microsoft, released August 2024. Its published size is 4.15B parameters. Licence: mit.

Catalogue identity notes

Comparison-only. Arena entry 'phi-3.5-vision-instruct' (vision). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'phi-3.5-vision-instruct' as 'Phi-3.5-vision-instruct' (Microsoft, MIT), linking https://huggingface.co/microsoft/Phi-3.5-vision-instruct. Maker repo microsoft/Phi-3.5-vision-instruct @12b77fb: licence tag mit. 4,146,621,440 parameters from the Hugging Face safetensors metadata (dense, so active = total). Released 2024-08: card: 'Release date: August 2024'.

37.4±1.6#570 of 719 LLMsfrom 21 benchmark results across 5 sources

$ how it compares

The LLMs ranked either side of Phi-3.5-vision-instruct, out of 828 with a score, then more from Microsoft.

#	model	score	where	RAM	released
661	Code Tulu 2 7B Allen Institute for AI	38±2	local	~16GB	2023-11
662	SOLAR 10.7B Instruct v1.0 Upstage	37±1	local	~8.3GB	2023-12
663	Phi-3.5-vision-instruct Microsoft	37±2	local	~10GB	2024-08
664	Tulu 2 DPO 13B Allen Institute for AI	37±1	local	~9.8GB	2023-11
665	Yi-1.5 6B (base) 01.AI	37±3	local	~5.1GB	2024-05
	more from Microsoft
252	Phi-4-reasoning-plus 14B Microsoft	59±1	local	~10GB	2025-04
261	Phi-4-reasoning 14B Microsoft	58±1	api $0.22/M +local	~10GB	2025-04
365	Phi-4 14B Microsoft	53±1	api $0.087/M +local	~10GB	2024-12

see all LLMs · find the ones that fit your machine

$ what it's good at

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #306 of 393 * · 5.0% of its score · 1 test

Beats Gemma 4 E4B, OpenThinker3 7B, Gemma 4 E2B here, though they rank higher overall. Trails LFM2 350M, Gemma 3 1B, Flan-T5 XXL here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
IFEval (aggregation unspecified)	54.3% ✎	▄···▄█▄····▄ beats 0/5	easy · avg 79%	▇▇ 5.0%	at its level

math▁▂▃▅▆▅▄█▄▁▁▁ #306 of 398 * · 3.0% of its score · 1 test

Beats GPT-4 (0613), Athene-70B, Llama 3 70B Instruct here, though they rank higher overall. Trails Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MathVista (testmini) lab	43.9% ✎	▁··▁▄▄▇█▅▆█▄ beats 3/47	medium · avg 55%	▇ 3.0%	noisy

vision & documents▃▁▃▂▁▃▄▂▆▇█▄ #144 of 147 · 72% of its score · 15 tests

Beats Gemma 3 4B, Phi-3-vision-128k-instruct here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
Arena vision (style controlled)	919.5 ✎	▁▃▂▂▂▂▄▅▅▆██ beats 1/145	avg 1068	▇▇▇▇▇▇ 15%	blunt
CharXiv descriptive questions	63.7% ✎	█·▄·······▄█ beats 2/5	easy · avg 78%	▇▇▇▇▇ 14%	at its level
MMStar	49.5% ✎	▄▄▄█·······▄ beats 0/5	medium · avg 62%	▇▇▇ 7.9%	at its level
MMMU (val) lab	43.0% ✎	▁····▁▁▄█▇▆▆ beats 4/52	medium · avg 52%	▇▇ 5.9%	typical
MMBench-EN dev v1.1	73.1% ✎	█·····▃··▃·▃ beats 3/5	easy · avg 79%	▇▇ 5.6%	blunt · at its level
HallusionBench	58.5% ✎	▄▄·▄·····█·▄ beats 0/5	medium · avg 65%	▇▇ 4.3%	blunt · at its level
MathVista (testmini) lab	43.9% ✎	▁··▁▄▄▇█▅▆█▄ beats 3/47	medium · avg 55%	▇ 3.0%	noisy
CountBench	64.5% ✎	▄··▄··█··▄·▄ beats 1/5	very easy · avg 86%	▇ 2.6%	at its level · noisy

7 more tests

test	result	vs the others in its run	difficulty	counts	why
Ai2 Molmo human preference Elo (Sept 2024, 27 VLMs)	982.0 ✎	▄▄·▄·▄▄▄·▄·█ beats 1/8	avg 1037	▇ 2.4%	blunt
RealWorldQA	58.0% ✎	▄···█··▄▄··▄ beats 1/5	easy · avg 69%	▇ 2.3%	blunt · at its level
PhysBench physical-world understanding (test, Avg)	39.8% ✎	▅▃█··▅▃····▃ beats 5/9	hard · avg 43%	▇ 2.3%	blunt
AI2D test +1	79.0% ✎	█···▄▄···▄·▄ beats 5/5	easy · avg 82%	▇ 1.8%	blunt · at its level · several runs
OCRBench	64.2% ✎	█··███·█···█ beats 0/5	easy · avg 81%	▇ 1.7%	blunt · at its level
ChartQA +1	82.1% ✎	▄······▄··██ beats 5/5	easy · avg 80%	▇ 1.5%	blunt · at its level · noisy · several runs
TextVQA lab	72.0% ✎	▁▁▁▄▅▃▃·▃▃▃█ beats 10/24	easy · avg 76%	▇ 1.3%	blunt · at its level

other tests3 not in a task · 22% of its score

test	result	vs the others in its run	difficulty	counts	why
MMLU	35.5% ✎	▄·····▄▄··▄█ beats 0/5	medium · avg 70%	▇▇▇▇ 10%	at its level
MMLU-Pro	28.6% ✎	█▄·······▄▄▄ beats 0/5	medium · avg 50%	▇▇▇ 8.5%	typical
Multi-IF (aggregation unspecified)	30.4% ✎	▄·▄··█▄····▄ beats 0/5	medium · avg 58%	▇▇ 4.4%	blunt · at its level

$ where to run it

Other ways to run it

Run it as published · needs ~8.29 GB, NVIDIA only · from github.com/vllm-project
vllm serve microsoft/Phi-3.5-vision-instruct --trust-remote-code

Official weights and documentation: huggingface.co/microsoft ↗

$ how the score was worked out · CohereLabs/North-Micro-Vision… 67%, LMArena leaderboard dataset 15%, Microsoft Phi-3.5-vision-inst… 12% and 2 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
CohereLabs/North-Micro-Vision-Instruct model card (benchmark table)	13	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 67%	Apache-2.0
LMArena leaderboard dataset	1	▇▇▇▇▇ 15%	CC BY 4.0
Microsoft Phi-3.5-vision-instruct model card: Benchmarks table · lab-reported	5	▇▇▇▇ 12%	MIT
Ai2 Molmo-72B-0924 model card: Human Preference Elo Rating column	1	▇ 2%	Apache-2.0
PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (arXiv:2501.16411v2)	1	▇ 2%	CC-BY-4.0
prior · pull toward the middle		▇ 1%
weighted average		= 37.4

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json