whatmodel.sh / Qwen2.5 72B Instruct

Qwen2.5 72B Instruct

+ contribute

Qwen2.5 72B Instruct is an open-weights language model from Alibaba, released September 2024. Its published size is 72B parameters and a 33k-token context window. Licence: qwen.

55.2±0.5#257 of 719 LLMsfrom 62 benchmark results across 30 sources

$ how it compares

The LLMs ranked either side of Qwen2.5 72B Instruct, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
307	GPT-4 Turbo (2024-04-09) OpenAI	55±0	api $15.00/M		2024-04
308	Yi-Lightning 01.AI	55±1	api price unknown		2024-10
309	Qwen2.5 72B Instruct Alibaba	55±1	api $0.37/M +local	~30GB	2024-09
310	Mistral Large 2 (2411) Mistral	55±1	local	~78GB	2024-11
311	Ministral 3 8B Reasoning 2512 Mistral	55±1	local	~6.5GB	2025-12
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #197 of 300 · 5.7% of its score · 6 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 (Jan '25) here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Mellum2 12B-A2.5B Instruct here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified	33.0% ✎	▃···▃·▃·▃█·▃ beats 5/7	hard · avg 19%	▇ 2.6%	sharp · at its level
ContractBench success rate (33 observation-contract tool tasks)	23.2% ✎	▆··▆▄▂▄▂▆▆█▄ beats 6/23	very hard · avg 14%	▇ 1.5%	sharp
BFCL v3	63.4% ✎	▁▁▁▁▁▃▅█▅▄▁▁ beats 90/114	medium · avg 52%	▇ 0.5%	blunt · at its level
Galileo Agent Leaderboard v2 (action completion)	51.0% ✎	▅▃·▃▃▅▅█▃█▅▅ beats 13/19	hard · avg 27%	▇ 0.4%	at its level
Multi-SWE-bench, 7 languages (MagentLess scaffold)	2.6% ✎	▄···▄▄·▄▄·▄█ beats 0/7	very hard · avg 2.6%	▇ 0.4%	blunt · too hard for it
Galileo Agent Leaderboard v1 (tool selection quality)	81.7% ✎	▃▃▃▃▆▅▆▅█▂▅▂ beats 11/31	easy · avg 79%	▇ 0.4%	blunt

coding▁▁▁▃▆██▇▆▄▃▁ #216 of 458 · 29% of its score · 16 tests

Beats Qwen3.5 4B, Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B) here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 3B Reasoning 2512, DeepSeek-Coder-V2 Instruct (0724) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LiveCodeBench official (Jan 1–Nov 1, 2024)	41.7% ✎	▁▁·▁▂▄▆█▅▂·▂ beats 26/31	hard · avg 26%	▇▇ 5.8%	sharp · at its level
CRUXEval (ZeroEval, 800 problems)	73.9% ✎	▁▂▆▃▆█▆▃▂·▁▁ beats 33/47	medium · avg 61%	▇▇ 4.9%	sharp
Aider code editing (133 Exercism Python exercises, two attempts)	65.4% ✎	▁·▁▂▄▁▆▄█▆▂▂ beats 37/62	medium · avg 54%	▇▇ 3.8%	at its level
SWE-bench Verified	33.0% ✎	▃···▃·▃·▃█·▃ beats 5/7	hard · avg 19%	▇ 2.6%	sharp · at its level
Aider polyglot (225 exercises, two attempts)	8.0% ✎	█▄··█·█····▄ beats 3/7	very hard · avg 6.7%	▇ 2.5%	sharp · noisy
HumanEval+	78.4% ✎	▁·▁▁▂▃▄▄█▅▄▃ beats 104/145	easy · avg 74%	▇ 2.0%	sharp
LiveCodeBench v5 (Oct 2024–Feb 2025)	26.3% ✎	▄▄·▄·····█▄█ beats 3/7	hard · avg 19%	▇ 1.6%	at its level
MBPP+	88.6% ✎	██·██···████ beats 6/7	easy · avg 79%	▇ 1.0%	typical

8 more tests

test	result	vs the others in its run	difficulty	counts	why
WebApp1K-React pass@1 (onekq-ai)	35.4% ✎	▃▁▁█▃▄▅▃▄▇▁▄ beats 10/31	medium · avg 44%	▇ 1.0%	sharp · at its level · noisy
BIRD-SQL (dev)	50.1% ✎	▄··▄·▄·█··▄█ beats 2/7	medium · avg 49%	▇ 0.8%	at its level
CodeReviewQA: change localisation (hard), invariant accuracy	58.3% ✎	█▂▁▁▂▂▂▃▅▅▅▇ beats 40/47	medium · avg 36%	▇ 0.8%	sharp · at its level · noisy
HumanEval (simple-evals)	80.4% ✎	▂▅▆█▅▃··▂··▂ beats 2/19	easy · avg 78%	▇ 0.6%	typical
CodeReviewQA: solution identification (hard), invariant accuracy	90.9% ✎	▃··▁▂▂▃▂█▃▄▃ beats 47/47	medium · avg 54%	▇ 0.5%	sharp · at its level · noisy
CodeReviewQA: automated code refinement, exact match rate	48.7% ✎	▁▁▁▁▁▁▂▂▃█▅▄ beats 46/47	hard · avg 33%	▇ 0.5%	at its level
Multi-SWE-bench, 7 languages (MagentLess scaffold)	2.6% ✎	▄···▄▄·▄▄·▄█ beats 0/7	very hard · avg 2.6%	▇ 0.4%	blunt · too hard for it
CodeReviewQA: change type recognition, invariant accuracy	79.8% ✎	▁▁▁······▁▃█ beats 47/47	easy · avg 67%	▇ 0.0%	blunt · at its level · noisy

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #212 of 393 · 15% of its score · 9 tests

Beats Granite 4.2 8B, Mistral Small 4 (119B-A6.5B), Gemma 2 9B IT SimPO here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, Granite 4.0 H Micro 3B, Gemma 3n E4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena-Hard (v0.1, GPT-4-1106 judge)	78.0% ✎	▂▁▁▅█▅▄▃▃▅▃▁ beats 50/60	medium · avg 52%	▇▇ 4.9%	sharp · at its level
Arena-Hard v2.0 hard prompts (style control, Gemini-2.5 judge)	10.1% ✎	▂·▃▅▅·▃█▆▂▂▂ beats 1/22	very hard · avg 12%	▇ 2.6%	sharp
LMArena (text)	1302.8 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 128/379	avg 1268	▇ 2.2%	blunt
Arabic IFEval prompt-level (Inception)	67.3% ✎	▁·▁▁▂▁▄▄▆▄█▄ beats 36/38	medium · avg 49%	▇ 1.8%	sharp · at its level
AlpacaEval 2 length-controlled	47.7% ✎	▃▃·▁▁█▇▅·▁·▁ beats 18/22	medium · avg 42%	▇ 1.5%	at its level
Arena-Hard (version/judge unspecified)	78.4% ✎	▃▃▂▅▃▆▅█▅▅▂▃ beats 25/30	easy · avg 66%	▇ 1.0%	at its level
Arena-Hard v2.0 creative writing (GPT-4.1 + Gemini-2.5 judges)	10.2% ✎	▆▂▂▆▂▂▃█▂▃·▂ beats 1/22	hard · avg 19%	▇ 0.5%	sharp · at its level · noisy
IFEval prompt-level loose	87.6% ✎	▁▃▁▃▁▄▁▁█▃▃▁ beats 20/22	easy · avg 80%	▇ 0.3%	typical

1 more test

test	result	vs the others in its run	difficulty	counts	why
IFEval (aggregation unspecified)	85.0% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 75/141	easy · avg 77%	▇ 0.3%	typical

math▁▂▃▅▆▅▄█▄▁▁▁ #221 of 398 · 12% of its score · 6 tests

Beats GPT-4o (2024-08-06), Command A (03-2025), GPT-4o (2024-11-20) here, though they rank higher overall. Trails Olmo 3 7B Instruct, DeepSeek R1 Distill Llama 8B, Phi-4-mini-reasoning 3.8B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH level 5	63.2% ✎	▂▂▅▆▄█▆▃▅▂▃▄ beats 53/92	hard · avg 32%	▇▇ 5.4%	sharp · at its level
MATH Level 5 (ZeroEval, 721 problems)	60.2% ✎	▁▁▁▆█▂▇▇▄··▂ beats 34/38	hard · avg 34%	▇ 3.4%	sharp · at its level
OTIS mock AIME 2024–2025	8.1% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 46/193	very hard · avg 8.5%	▇ 1.5%	sharp · noisy
MATH (subset unspecified)	75.9% ✎	▃▃█▂▂▆·····▁ beats 19/22	medium · avg 61%	▇ 0.7%	at its level
GSM8K	89.5% ✎	▂▃··▃▂▅█▆▃▂▃ beats 18/22	very easy · avg 88%	▇ 0.7%	sharp · maxed out
MGSM (simple-evals)	87.3% ✎	▁·▁▁·▁▇▃█▁▃▁ beats 8/20	easy · avg 82%	▇ 0.5%	blunt

other tests27 not in a task · 41% of its score

test	result	vs the others in its run	difficulty	counts	why
MMLU-Redux (ZeroEval, 2,778 questions)	85.6% ✎	▁·▁▁▄▄▄▅█▄▂▁ beats 45/53	easy · avg 77%	▇▇▇ 7.6%	typical
ZebraLogic (ZeroEval, 1,000 puzzles)	26.6% ✎	▁▁▃▆██▁▁·▁▁▁ beats 41/57	hard · avg 18%	▇▇ 4.9%	sharp · at its level
GPQA Diamond	49.1% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 68/208	hard · avg 40%	▇ 3.6%	sharp · at its level
BIG-Bench Hard	80.4% ✎	▁▄·▁▁·▅█·▄▄▁ beats 17/22	easy · avg 75%	▇ 2.2%	sharp
GPQA Diamond (simple-evals)	49.0% ✎	▁·····▄█▂▂▃▃ beats 6/22	hard · avg 41%	▇ 2.2%	sharp · at its level
MMLU	85.5% ✎	▁▂·▁·▁▁█·▃▃▅ beats 20/22	easy · avg 77%	▇ 2.2%	typical
MMLU-Pro	69.6% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 87/188	medium · avg 59%	▇ 1.5%	at its level
RepoQA (32K, threshold 0.8)	83.2% ✎	▄▄▄··▄▄··▄·█ beats 3/7	easy · avg 81%	▇ 1.4%	typical

19 more tests

test	result	vs the others in its run	difficulty	counts	why
LiveBench 2024-08-31	52.9% ✎	▄█·▄·▄▄·▄··▄ beats 4/7	medium · avg 42%	▇ 1.4%	at its level
EQ-Bench 3 (emotional intelligence, judge Elo)	1076.0 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 9/62	avg 1044	▇ 1.4%	typical
CPTUB average (SpeakLeash, 0-5 judge scale)	78.9% ✎	▁▂▂▂▃█▃▆▅▄▂▃ beats 79/95	easy · avg 66%	▇ 1.3%	at its level
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	48.9% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 41/66	hard · avg 31%	▇ 1.2%	at its level
BRIDGE clinical text, zero-shot average (87 tasks)	41.6% ✎	▁▁▁▁▁▂▂▃▄▆█▃ beats 83/97	hard · avg 32%	▇ 1.2%	blunt · at its level
BigCodeBench (Instruct)	43.6% ✎	▄··▄▄···▄▄█▄ beats 4/7	medium · avg 36%	▇ 1.1%	at its level
EvalPerf correctness (118 tasks, 20240328)	88.3% ✎	▂·▂▂▂▆▅▆▆▆▃█ beats 27/29	easy · avg 76%	▇ 1.1%	typical
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	23.0% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 7/39	very hard · avg 22%	▇ 1.0%	blunt
Polish EQ-Bench v2 (SpeakLeash)	68.9% ✎	▁··▁▁▂▂▆▅▆█▆ beats 49/70	medium · avg 64%	▇ 1.0%	at its level
Open Persian LLM Leaderboard v2 average (21 tasks)	65.6% ✎	▂▄▂▃█▄▃▆▃▅▂▂ beats 65/81	medium · avg 58%	▇ 0.9%	blunt · at its level
MMLU (simple-evals, zero-shot CoT)	85.3% ✎	▁···▂▁▅█▂▂▃▁ beats 7/22	easy · avg 80%	▇ 0.7%	typical
PopQA	30.6% ✎	▁▁▁▃▃▃▁█▁▃·▄ beats 14/21	hard · avg 32%	▇ 0.7%	blunt · at its level
SurgiQ surgical MCQ (zero-shot, log-probability scoring)	68.0% ✎	▂···▂▂·▃▂▃██ beats 17/17	medium · avg 53%	▇ 0.5%	at its level
DiagnosticIQ industrial maintenance MCQ (macro accuracy, 4 options)	61.2% ✎	▁··▃▁▁▁▅█▅▁▅ beats 11/24	medium · avg 52%	▇ 0.4%	blunt · at its level
TruthfulQA MC2	69.9% ✎	▃▃▂·▃▃·▅█▂▅▂ beats 18/21	medium · avg 63%	▇ 0.4%	blunt · at its level
Open PT-LLM Leaderboard average (9 Portuguese tasks)	81.2% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 264/276	medium · avg 73%	▇ 0.4%	at its level · noisy
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy	96.3% ✎	▁▁▁▁▁▁▁▂▄█▄▃ beats 146/146	easy · avg 83%	▇ 0.2%	noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	95.7% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 80/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy
DROP F1, 3-shot (simple-evals)	76.7% ✎	▅▂▆█▆·▃·▂▅·▂ beats 2/23	easy · avg 80%	▇ 0.1%	blunt

$ where to run it

Through an API

api · DeepInfra · $0.37/M · $0.36 in / $0.4 out per 1M tokens · 33k context · deepinfra.com ↗ · price sourceUSD per 1M text tokens, standard uncached input and output. Product: Serverless text generation. API model: Qwen/Qwen2.5-72B-Instruct. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen2.5 72B Instruct · needs ~44 GB · from huggingface.co/Qwen (official)
chat in the terminal
llama cli -hf Qwen/Qwen2.5-72B-Instruct-GGUF:q4_k_m
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf Qwen/Qwen2.5-72B-Instruct-GGUF:q4_k_m

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen2.5 72B Instruct · needs ~44 GB · from huggingface.co/Qwen (official)
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/Qwen/Qwen2.5-72B-Instruct-GGUF@q4_k_m -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen2.5 72B Instruct · needs ~43.7 GB · from ollama.com
chat in the terminal
ollama run qwen2.5:72b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen2.5 72B Instruct · needs ~40.9 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Qwen2.5-72B-Instruct-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Qwen2.5-72B-Instruct-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · WildEval ZeroEval published r… 21%, Cohere — Command A technical… 11%, Epoch AI — internal evaluatio… 11% and 27 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
WildEval ZeroEval published results	4	▇▇▇▇▇▇ 21%	Apache-2.0
Cohere — Command A technical report	7	▇▇▇ 11%	CC BY 4.0
Epoch AI — internal evaluations	3	▇▇▇ 11%	CC BY 4.0
Microsoft Research — Phi-4 Technical Report	9	▇▇▇ 9%	CC BY 4.0
2 OLMo 2 Furious (Team OLMo, Ai2) — authors' own evaluations of OLMo 2 Instruct and peer instruct models	8	▇▇▇ 9%	CC BY 4.0
LiveCodeBench official website results (2024-12-26)	1	▇▇ 6%	CC-BY-SA-4.0
Arena-Hard-Auto v0.1 leaderboard (repository README, pre-v2)	1	▇ 5%	Apache-2.0
Aider code editing benchmark runs (old edit leaderboard)	1	▇ 4%	Apache-2.0
Arena-Hard-Auto v2.0-Preview leaderboard (repository README)	2	▇ 3%	Apache-2.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	2	▇ 3%	Apache-2.0
LMArena leaderboard dataset	1	▇ 2%	CC BY 4.0
CodeReviewQA: The Code Review Comprehension Assessment for Large Language Models (arXiv:2503.16167v2)	4	▇ 2%	CC-BY-4.0
ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)	1	▇ 2%	CC BY 4.0
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets	1	▇ 1%	Apache-2.0 (stated in the LiveBench datasheet)
EQ-Bench 3 canonical leaderboard results	1	▇ 1%	MIT
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 1%	MIT
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)	1	▇ 1%	MIT
EvalPlus / EvalPerf published evaluation runs	1	▇ 1%	Apache-2.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 1%	CC BY 4.0
SpeakLeash Polish EQ-Bench leaderboard	1	▇ 1%	Apache-2.0
WebApp1K models leaderboard (onekq-ai)	1	▇ 1%	MIT
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)	1	▇ 1%	Apache-2.0
Galileo Agent Leaderboard (v1 and v2 results)	2	▇ 1%	Apache-2.0
SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)	1	▇ 1%	CC BY 4.0
Berkeley Function Calling Leaderboard result archive	1	▇ 0%	Apache-2.0
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)	1	▇ 0%	CC BY 4.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 0%	Apache-2.0
Multi-SWE-bench experiments (ByteDance Seed baseline runs)	1	▇ 0%	Apache-2.0
EuroEval leaderboards (results collection), English generative leaderboard	1	▇ 0%	MIT
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 55.2