whatmodel.sh / Qwen3-Next 80B-A3B Thinking

Qwen3-Next 80B-A3B Thinking

+ contribute

Qwen3-Next 80B-A3B Thinking is an open-weights language model from Alibaba, released September 2025. Its published size is 80B parameters (3B active per token) and a 131k-token context window. Licence: apache-2.0.

63.2±0.7#163 of 719 LLMsfrom 52 benchmark results across 9 sources

$ how it compares

The LLMs ranked either side of Qwen3-Next 80B-A3B Thinking, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
186	MiniCPM5 2B OpenBMB	64±1	local	~3GB	2026-09
187	Grok 3 Beta SpaceXAI (xAI)	63±1	comparison only		2025-04
188	Qwen3-Next 80B-A3B Thinking Alibaba	63±1	api $0.41/M +local	~27GB	2025-09
189	DeepSeek R1 (Jan '25) DeepSeek	63±1	api $4.00/M +local	~110GB	2025-01
190	Gemini 2.5 Flash Preview (09-2025) Google	63±1	comparison only		2025-09
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #147 of 300 · 16% of its score · 8 tests

Beats o4-mini, o1 (2024-12-17), o3-mini here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
τ²-bench Telecom lab	43.9% ✎	▁·▅█▆▅▂▃▃▂▃▁ beats 37/89	very hard · avg 9.5%	▇▇ 5.3%	sharp · at its level
τ-bench Retail lab	69.6% ✎	▁····▃·▂▇█▂▆ beats 20/28	medium · avg 36%	▇ 2.5%	at its level
τ²-bench Retail lab	67.8% ✎	▁····▂▁▃▆█▃▁ beats 23/44	medium · avg 37%	▇ 2.1%	at its level
τ-bench Airline lab	49.0% ✎	▁▁▁·▅▄▁█▇▁▁▇ beats 20/28	hard · avg 23%	▇ 1.7%	at its level
τ²-bench Airline lab	60.5% ✎	▁▁·▂▁▃▂▁█▅▂▃ beats 33/42	hard · avg 32%	▇ 1.6%	at its level
τ²-bench aggregate lab	57.4% ✎	▁▁▁·▂▂▄▃▃▅█▂ beats 8/37	medium · avg 37%	▇ 0.9%	at its level · noisy
BFCL V4 (December 2025 archive) lab	49.7% ✎	▁▁▁▂▇▆▆▆█▂▄▂ beats 71/111	medium · avg 36%	▇ 0.9%	at its level
BFCL v3 lab	72.0% ✎	▁▁▁▁▁▃▅█▆▅▄▁ beats 159/170	medium · avg 59%	▇ 0.6%	blunt · at its level

coding▁▁▁▃▆██▇▆▄▃▁ #139 of 458 * · 3.2% of its score · 1 test

Beats Gemini 2.5 Pro, Gemini 3.1 Flash-Lite, Qwen3-Coder-Next (80B-A3B) here, though they rank higher overall. Trails Devstral Small 2 24B, Llama 3.1 Nemotron Ultra 253B v1, Nemotron 3.5 Lightning 30B-A3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LiveCodeBench v6 (Feb–May 2025) lab +1	68.7% ✎	▁▁▃▁█▅▄▇▅▅▃█ beats 19/38	medium · avg 40%	▇ 3.2%	at its level · several runs

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #137 of 393 · 8.8% of its score · 3 tests

Beats o3-mini, EXAONE 4.5 33B, INTELLECT-3 here, though they rank higher overall. Trails LFM2.5 8B-A1B, LFM2.5 2.6B, Spark-X2.5 1.7B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena-Hard v2 aggregate lab	62.3% ✎	▁▁··▄▃▁▃█▁▁▇ beats 11/22	hard · avg 28%	▇▇ 4.0%	sharp · at its level
LMArena (text)	1369.4 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 212/379	avg 1268	▇▇ 3.8%	blunt
IFEval (aggregation unspecified) lab	88.9% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 103/141	easy · avg 78%	▇ 1.0%	typical

math▁▂▃▅▆▅▄█▄▁▁▁ #125 of 398 · 13% of its score · 5 tests

Beats Claude Sonnet 4.6, MiniMax-M3, MiMo-V2.5 (310B-A15B) here, though they rank higher overall. Trails Olmo 3.1 32B Think, Granite 4.2 8B, Falcon-H1R 7B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
HMMT 2025 (edition unspecified) lab	73.9% ✎	▃█▇▅▇▅▁·▁··▁ beats 20/32	hard · avg 33%	▇▇ 4.3%	sharp · at its level
PolyMATH lab +1	56.3% ✎	▁···▃▁·▇█▄▄▄ beats 18/23	hard · avg 32%	▇ 3.0%	at its level · several runs
AIME 2025 lab	87.8% ✎	▁·▁▁▁▂▅▆█▇▅▁ beats 106/133	medium · avg 51%	▇ 2.4%	sharp · small run
HMMT February 2025 lab	73.7% ✎	▁▁▁▃▁▅▅▃██▁▄ beats 11/31	medium · avg 48%	▇ 2.0%	typical
HMMT November 2025 lab	81.2% ✎	▁▁····▁▅▃█▇▁ beats 6/23	medium · avg 61%	▇ 1.7%	typical

other tests30 not in a task · 59% of its score

test	result	vs the others in its run	difficulty	counts	why
Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations)	0.5888 ✎	▁▄▃▃▇▄█▃▅▃▁▁ beats 28/30	avg 0.423	▇▇ 5.9%	sharp
GPQA Diamond lab	77.2% ✎	▁▁▁▁▂▄▅▅███▄ beats 185/310	hard · avg 48%	▇▇ 4.9%	sharp · at its level
GPQA (subset unspecified) lab	77.2% ✎	▂▁▂▂▄▃▆▃▅█▄▃ beats 90/104	hard · avg 49%	▇▇ 4.2%	sharp · at its level
INCLUDE lab +1	78.9% ✎	▂····▂▂▄▄▄█▄ beats 24/36	medium · avg 58%	▇▇ 4.1%	at its level · several runs
MMLU-Pro lab +1	82.7% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 152/188	medium · avg 61%	▇▇ 3.8%	several runs
Medmarks-V weighted mean win rate (61 models, 71 configurations)	58.9% ✎	▂▃▅▂▆▃█▅█▅▃▆ beats 29/34	medium · avg 42%	▇ 2.9%	blunt · at its level
MMLU-ProX lab +1	78.7% ✎	▁·▁▁▁▂▁▃▂█▅▄ beats 19/35	medium · avg 60%	▇ 2.8%	several runs
LongBench v2 lab	48.0% ✎	▂·▁·▂▂▁▃▄▁█▄ beats 14/28	very hard · avg 35%	▇ 2.5%	at its level

22 more tests

test	result	vs the others in its run	difficulty	counts	why
C-Eval lab	89.7% ✎	▁···▁▁▂▁▂▂▃█ beats 21/33	medium · avg 73%	▇ 2.4%	typical
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	28.3% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 15/39	very hard · avg 22%	▇ 2.4%	blunt
BRIDGE clinical text, zero-shot average (87 tasks)	43.9% ✎	▁▁▁▁▁▂▂▃▄▆█▃ beats 90/97	hard · avg 32%	▇ 2.2%	blunt · at its level
MMMLU lab	81.3% ✎	▂▁▁▂▃▂▄▄▅██▃ beats 32/46	medium · avg 63%	▇ 2.0%	at its level
OJBench lab	29.7% ✎	▁···▁▃·▅▄▅██ beats 16/26	very hard · avg 13%	▇ 2.0%	at its level
MMLU-Redux lab	92.5% ✎	▁▁▃▁▂▃▄▅▇▅█▄ beats 47/63	easy · avg 82%	▇ 2.0%	typical
Global PIQA lab	83.5% ✎	▃▃··▃··▃▃▅█▅ beats 6/11	medium · avg 71%	▇ 1.7%	at its level
VitaBench lab	29.5% ✎	▄··▂▄▆▆█▄▄▄▂ beats 13/21	very hard · avg 10%	▇ 1.7%	at its level
MultiChallenge lab	51.3% ✎	▁·▁·▁▃▆▅▃▁█▅ beats 15/29	hard · avg 30%	▇ 1.5%	at its level
MAXIFE lab	79.9% ✎	▂·▂·▂···▆·▄█ beats 5/11	medium · avg 63%	▇ 1.5%	typical
SuperGPQA lab	60.8% ✎	▂▁▂▂▅▆▃▇▇▇█▃ beats 36/46	medium · avg 39%	▇ 1.5%	at its level
IFBench (aggregation unspecified) lab	61.5% ✎	▃▇▂▄▅▅▂▄▄▆█▇ beats 45/80	medium · avg 39%	▇ 1.5%	at its level
WritingBench lab	84.6% ✎	▁·▁▂▁··▅█▂▂▁ beats 9/20	easy · avg 70%	▇ 1.3%	typical
LiveBench 2024-11-25 lab	76.6% ✎	▄··▂·▂·▂█▆▂▂ beats 9/13	medium · avg 60%	▇ 1.1%	typical
Multi-IF (aggregation unspecified) lab	77.8% ✎	▁▁▁▁▁▁▃▄█▄▁▂ beats 27/41	medium · avg 63%	▇ 1.1%	blunt
NOVA-63 lab	53.3% ✎	▃▃·▃···▃█▃▃█ beats 4/11	medium · avg 38%	▇ 1.0%	blunt · at its level
DeepPlanning lab	0.4% ✎	▃·····▂··▄█▄ beats 0/9	very hard · avg 7.1%	▇ 0.4%	noisy
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy	88.1% ✎	▃·▁▁▂▁▂▅█▇▄▁ beats 87/100	medium · avg 68%	▇ 0.4%	noisy
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy	95.9% ✎	▁▁▁▁▁▁▁▂▄█▄▃ beats 145/146	easy · avg 83%	▇ 0.2%	noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)	96.3% ✎	▁▁▂▃▄▄█▆▃▂▂▁ beats 94/140	very easy · avg 94%	▇ 0.1%	blunt · maxed out · noisy
WMT24++ lab	57.4% ✎	▁·▁▁·▃·▁▂▆█▃ beats 3/27	easy · avg 68%	▇ 0.1%	blunt · at its level
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	90.7% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 52/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · Alibaba Cloud Model Studio · $0.41/M · $0.15 in / $1.2 out per 1M tokens · 131k context · www.alibabacloud.com ↗ · price sourceUSD per 1M tokens, Singapore (International), qwen3-next-80b-a3b-thinking, thinking mode only. Observed 2026-09-25.

llama.cpp · recommended

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3-Next 80B-A3B Thinking · needs ~24.5 GB · from huggingface.co/unsloth
chat in the terminal
llama cli -hf unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF:UD-Q2_K_XL
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF:UD-Q2_K_XL

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3-Next 80B-A3B Thinking · needs ~44.8 GB, Mac only · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Qwen3-Next-80B-A3B-Thinking-MLX-4bit -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3-Next 80B-A3B Thinking · needs ~48.5 GB · from ollama.com
chat in the terminal
ollama run qwen3-next:80b-a3b-thinking-q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3-Next 80B-A3B Thinking · needs ~44.8 GB, Mac only · from huggingface.co/lmstudio-community
chat in the terminal
mlx_lm.chat --model lmstudio-community/Qwen3-Next-80B-A3B-Thinking-MLX-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Qwen3-Next-80B-A3B-Thinking-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3-Next-80B-A3B-Think… 46%, Qwen/Qwen3.5-9B model card 26%, Qwen/Qwen3.5-4B model card 9% and 6 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Qwen/Qwen3-Next-80B-A3B-Thinking model card · lab-reported	22	▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 46%	apache-2.0
Qwen/Qwen3.5-9B model card · lab-reported	18	▇▇▇▇▇▇▇▇ 26%	apache-2.0
Qwen/Qwen3.5-4B model card · lab-reported	3	▇▇▇ 9%	apache-2.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models	2	▇▇▇ 9%	CC BY 4.0
LMArena leaderboard dataset	1	▇ 4%	CC BY 4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 2%	CC BY 4.0
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)	1	▇ 2%	MIT
EuroEval leaderboards (results collection), English generative leaderboard	2	▇ 1%	MIT
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	2	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 63.2

$ quantization evidence · 26 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
FP8 (Qwen official checkpoint Qwen/Qwen3-Next-80B-A3B-Thinking-FP8; recipe not described in this card)	GPQA Diamond (reasoning table; per sibling FP8-dynamic card: pass@1 over n=8, lighteval, vLLM)	0.737 → 0.768	104.1%	source ↗
FP8 (Qwen official checkpoint Qwen/Qwen3-Next-80B-A3B-Thinking-FP8; recipe not described in this card)	MATH-500 (reasoning table; per sibling FP8-dynamic card: pass@1 over n=8, lighteval, vLLM)	0.94 → 0.876	93.2%	source ↗
FP8 (Qwen official checkpoint Qwen/Qwen3-Next-80B-A3B-Thinking-FP8; recipe not described in this card)	AIME 2025 (reasoning table; per sibling FP8-dynamic card: pass@1 over n=8, lighteval, vLLM)	0.6 → 0.5	83.3%	source ↗
FP8 block (E4M3; weights static 128x128 blocks, activations dynamic per-group 128; llm-compressor FP8_BLOCK, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	GPQA Diamond (reasoning table; per sibling FP8-dynamic card: pass@1 over n=8, lighteval, vLLM)	0.737 → 0.768	104.1%	source ↗
FP8 block (E4M3; weights static 128x128 blocks, activations dynamic per-group 128; llm-compressor FP8_BLOCK, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MATH-500 (reasoning table; per sibling FP8-dynamic card: pass@1 over n=8, lighteval, vLLM)	0.94 → 0.944	100.4%	source ↗
FP8 block (E4M3; weights static 128x128 blocks, activations dynamic per-group 128; llm-compressor FP8_BLOCK, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	AIME 2025 (reasoning table; per sibling FP8-dynamic card: pass@1 over n=8, lighteval, vLLM)	0.6 → 0.5	83.3%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	GPQA Diamond pass@1 (n=8; lighteval, vLLM)	0.737 → 0.748	101.4%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	GSM8K 5-shot strict-match (OpenLLM v1 configuration; lm-evaluation-harness, vLLM)	0.846 → 0.848	100.2%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MMLU 5-shot acc (OpenLLM v1 configuration; lm-evaluation-harness, vLLM)	0.85 → 0.851	100.1%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MMLU-Pro 5-shot acc (OpenLLM v2 leaderboard configuration; lm-evaluation-harness, vLLM)	0.727 → 0.728	100.1%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	OpenLLM v1 average (6 tasks: ARC-Challenge 25-shot acc_norm, GSM8K 5-shot strict-match, HellaSwag 10-shot acc_norm, MMLU 5-shot, TruthfulQA mc2 0-shot, Winogrande 5-shot; lm-evaluation-harness, vLLM)	0.732 → 0.732	100.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	OpenLLM v2 average (6 tasks: IFEval inst-level strict, BBH 3-shot, Math-Hard 4-shot, GPQA 0-shot, MuSR 0-shot, MMLU-Pro 5-shot; lm-evaluation-harness leaderboard, vLLM)	0.391 → 0.384	98.3%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	IFEval inst-level strict acc 0-shot (OpenLLM v2 leaderboard configuration; lm-evaluation-harness, vLLM)	0.448 → 0.424	94.7%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	Reasoning average (3 tasks: AIME 2025, MATH-500, GPQA Diamond; pass@1 over n=8 samples; lighteval, vLLM)	0.759 → 0.705	92.9%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MATH-500 pass@1 (n=8; lighteval, vLLM)	0.94 → 0.868	92.3%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	AIME 2025 pass@1 (n=8; lighteval, vLLM)	0.6 → 0.5	83.3%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	IFEval inst-level strict acc 0-shot (OpenLLM v2 leaderboard configuration; lm-evaluation-harness, vLLM)	0.448 → 0.46	102.7%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	GPQA Diamond pass@1 (n=8; lighteval, vLLM)	0.737 → 0.737	100.0%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	OpenLLM v1 average (6 tasks: ARC-Challenge 25-shot acc_norm, GSM8K 5-shot strict-match, HellaSwag 10-shot acc_norm, MMLU 5-shot, TruthfulQA mc2 0-shot, Winogrande 5-shot; lm-evaluation-harness, vLLM)	0.732 → 0.729	99.6%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MMLU 5-shot acc (OpenLLM v1 configuration; lm-evaluation-harness, vLLM)	0.85 → 0.846	99.5%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	OpenLLM v2 average (6 tasks: IFEval inst-level strict, BBH 3-shot, Math-Hard 4-shot, GPQA 0-shot, MuSR 0-shot, MMLU-Pro 5-shot; lm-evaluation-harness leaderboard, vLLM)	0.391 → 0.389	99.5%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	GSM8K 5-shot strict-match (OpenLLM v1 configuration; lm-evaluation-harness, vLLM)	0.846 → 0.838	99.0%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MMLU-Pro 5-shot acc (OpenLLM v2 leaderboard configuration; lm-evaluation-harness, vLLM)	0.727 → 0.714	98.3%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	MATH-500 pass@1 (n=8; lighteval, vLLM)	0.94 → 0.872	92.8%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	Reasoning average (3 tasks: AIME 2025, MATH-500, GPQA Diamond; pass@1 over n=8 samples; lighteval, vLLM)	0.759 → 0.703	92.6%	source ↗
INT4 W4A16 (GPTQ, symmetric, group 128; llm-compressor, compressed-tensors); lm_head, MoE router/shared-expert gates and linear-attention (Gated DeltaNet) layers unquantized	AIME 2025 pass@1 (n=8; lighteval, vLLM)	0.6 → 0.5	83.3%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json