whatmodel.sh / Qwen3.6 35B-A3B

Qwen3.6 35B-A3B

+ contribute

Qwen3.6 35B-A3B is an open-weights language model from Alibaba, released April 2026. Its published size is 35.5B parameters (3B active per token) and a 262k-token context window. Licence: apache-2.0.

67.7±0.5#106 of 719 LLMsfrom 88 benchmark results across 10 sources

$ how it compares

The LLMs ranked either side of Qwen3.6 35B-A3B, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
119	Grok 4 (0709) SpaceXAI (xAI)	68±1	comparison only		2025-07
120	MiniMax-M2.7 MiniMax	68±1	api $0.53/M +local	~75GB	2026-03
121	Qwen3.6 35B-A3B Alibaba	68±1	api $0.31/M +local	~23GB	2026-04
122	Ornith 1.0 35B (35B-A3B) DeepReinforce	68±1	local	~23GB	2026-06
123	Hy3 preview (295B-A21B) Tencent	68±1	local	~629GB	2026-04
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #88 of 300 · 22% of its score · 12 tests

Beats Grok 4.20 (0309 reasoning), GPT-5.1, Grok 4.3 here, though they rank higher overall. Trails Hy3 preview (295B-A21B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified +2	64.4% ✎	▄▄·····▄▄▄▄█ beats 5/7	hard · avg 17%	▇▇▇ 6.4%	sharp · at its level · several runs
SWE-bench Multilingual +2	57.0% ✎	▄·▄·····█▄█▄ beats 5/7	hard · avg 18%	▇ 3.4%	at its level · several runs
Terminal-Bench 2.1	34.8% ✎	▄······█▄▄█▄ beats 5/7	very hard · avg 7.1%	▇ 2.7%	at its level
SWE-bench Pro (split unspecified)	40.6% ✎	▄···▄···▄▄██ beats 5/7	very hard · avg 6.5%	▇ 2.1%	sharp · at its level
SWE-bench Pro public set, Qwen-refined lab +1	49.5% ✎	▄······▄█·▄▄ beats 1/5	hard · avg 18%	▇ 1.8%	at its level · several runs
K-BrowseComp-Verified (Korean web-browsing agent questions, pass@1)	12.0% ✎	▃····█··▃█·▃ beats 3/8	very hard · avg 3.4%	▇ 1.2%	typical
Terminal-Bench 2.0 lab	51.5% ✎	▂▂▅▅████▆▂▃▃ beats 21/36	very hard · avg 14%	▇ 1.1%	at its level · small run
MCP-Atlas (public set) lab	62.8% ✎	▂·▁▂▁·▁▃█▆▆▂ beats 14/37	hard · avg 29%	▇ 0.9%	at its level · small run

4 more tests

test	result	vs the others in its run	difficulty	counts	why
MCPMark lab	37.0% ✎	▃·····█··▃█▃ beats 7/8	very hard · avg 10%	▇ 0.8%	at its level · small run
Toolathlon (Tool Decathlon) lab	26.9% ✎	▂▂▄▂▄▄▆▂█▆██ beats 3/27	very hard · avg 11%	▇ 0.8%	at its level · small run
WideSearch lab	60.1% ✎	█·▂▆··▂···▂▂ beats 2/10	hard · avg 34%	▇ 0.6%	at its level · small run
τ³-bench lab	67.2% ✎	▁▁▂▁·▁▁·▁▁▂█ beats 15/25	medium · avg 38%	▇ 0.5%	blunt · at its level · small run

coding▁▁▁▃▆██▇▆▄▃▁ #90 of 458 · 19% of its score · 7 tests

Beats Grok 4.20 (0309 reasoning), GPT-5.1, Grok 4.3 here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Qwen3 235B-A22B Thinking 2507 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified +2	64.4% ✎	▄▄·····▄▄▄▄█ beats 5/7	hard · avg 17%	▇▇▇ 6.4%	sharp · at its level · several runs
SWE-bench Multilingual +2	57.0% ✎	▄·▄·····█▄█▄ beats 5/7	hard · avg 18%	▇ 3.4%	at its level · several runs
LiveCodeBench v6 (Feb–May 2025) lab +1	80.4% ✎	▁▁▃▁█▅▄▇▅▅▃█ beats 31/38	medium · avg 47%	▇ 2.1%	several runs
SWE-bench Pro (split unspecified)	40.6% ✎	▄···▄···▄▄██ beats 5/7	very hard · avg 6.5%	▇ 2.1%	sharp · at its level
Luau Leetcode fully solved (Torpedo Software)	50.0% ✎	▁▁▁▁·▁▃▃▅▃█▇ beats 8/49	hard · avg 18%	▇ 1.8%	at its level
SWE-bench Pro public set, Qwen-refined lab +1	49.5% ✎	▄······▄█·▄▄ beats 1/5	hard · avg 18%	▇ 1.8%	at its level · several runs
SciCode (level unspecified)	37.5% ✎	▄···▄▄█▄·▄·▄ beats 6/7	very hard · avg 13%	▇ 1.0%	at its level

math▁▂▃▅▆▅▄█▄▁▁▁ #128 of 398 · 37% of its score · 13 tests

Beats Claude Sonnet 4.6, MiniMax-M3, MiMo-V2.5 (310B-A15B) here, though they rank higher overall. Trails Olmo 3.1 32B Think, Qwen3-VL 30B-A3B Thinking, Granite 4.2 8B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
FrontierMath tiers 1–3 v2	17.5% ✎	▂··▁▁▃▄▄█▅▃▃ beats 10/82	very hard · avg 1.2%	▇▇▇ 8.5%	sharp · at its level
MathArena ArXivMath June 2026 (48 problems)	27.1% ✎	▂···▄·██▆▆▄▂ beats 1/19	very hard · avg 4.6%	▇▇ 5.5%	sharp · at its level
HMMT February 2026 lab +1	83.6% ✎	▃▃▂▃·██▆█▅▃▅ beats 23/33	medium · avg 36%	▇▇ 4.6%	sharp · several runs
MathArena ArXivMath April 2026 (40 problems)	33.8% ✎	▁·····▅██▄·▁ beats 2/20	very hard · avg 7.2%	▇▇ 4.4%	sharp · at its level
AIME 2026 lab +1	92.7% ✎	▁▁▁▂▁▂▆█▆▂·▁ beats 33/49	medium · avg 58%	▇▇ 4.1%	sharp · maxed out · small run · several runs
IMO-AnswerBench lab +1	78.9% ✎	▁▁·▁▁▁▄▅█▄▂▃ beats 19/42	medium · avg 36%	▇ 3.1%	small run · several runs
MathArena ArXivMath May 2026 (40 problems)	37.5% ✎	▁··▃▄█▄▁·▃·▁ beats 1/17	very hard · avg 10%	▇ 1.7%	at its level
OTIS mock AIME 2024–2025	86.7% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 134/193	very hard · avg 8.5%	▇ 1.4%	sharp · noisy

5 more tests

test	result	vs the others in its run	difficulty	counts	why
HMMT February 2025 lab	90.7% ✎	▁▁▁▃▁▅▅▃██▁▄ beats 19/31	medium · avg 62%	▇ 1.1%	small run
MathArena BrokenArXiv June 2026 (54 problems)	3.4% ✎	▄··█▂▂▂▆▄··▂ beats 1/14	very hard · avg 1.1%	▇ 0.7%	sharp · noisy
MathVista (testmini) lab	86.4% ✎	▁··▁▄▄▇█▅▆█▄ beats 43/47	medium · avg 63%	▇ 0.7%	small run
HMMT November 2025 lab	89.1% ✎	▁▁····▁▅▃█▇▁ beats 10/23	easy · avg 66%	▇ 0.7%	small run
DynaMath lab	82.8% ✎	▂·····▂··█▄▆ beats 2/10	medium · avg 62%	▇ 0.6%	small run

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #55 of 147 · 6.0% of its score · 15 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5, Claude Opus 4.5 here, though they rank higher overall. Trails Qwen3.6 Flash, Qwen3.5 35B-A3B, Gemma 4 31B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv (reasoning questions) lab	78.0% ✎	▂▅▃▅▅▆▆▅▆█▆▂ beats 71/100	medium · avg 42%	▇ 0.8%	small run
MathVista (testmini) lab	86.4% ✎	▁··▁▄▄▇█▅▆█▄ beats 43/47	medium · avg 63%	▇ 0.7%	small run
MMMU (val) lab	81.7% ✎	▁····▁▁▄█▇▆▆ beats 48/52	medium · avg 57%	▇ 0.7%	small run
OmniDocBench 1.5 overall (higher is better) lab	89.9% ✎	▂··▂▂▂··▂▂█▅ beats 10/13	easy · avg 70%	▇ 0.7%	small run
MMMU Pro lab	75.3% ✎	▂▃▁▃▆▇▃▇▇██▅ beats 40/52	medium · avg 46%	▇ 0.6%	at its level · small run
MMStar lab	80.7% ✎	▃▅▂▃▂▂▆█▆▆█▅ beats 29/34	medium · avg 62%	▇ 0.5%	small run
CountBench lab	96.1% ✎	▂▂▃▂·▂▂▃▃·▅█ beats 11/18	very easy · avg 90%	▇ 0.4%	maxed out · small run
SimpleVQA lab	58.9% ✎	▃·▃·▃·▃·█▅▃▃ beats 7/10	hard · avg 34%	▇ 0.4%	at its level · small run

7 more tests

test	result	vs the others in its run	difficulty	counts	why
MMBench-EN dev v1.1 lab	92.8% ✎	█▄···▂··▄▂▄▆ beats 12/14	easy · avg 82%	▇ 0.3%	blunt · small run
HallusionBench lab	69.8% ✎	▃▁·▃▃▃▄▅▄█▅▅ beats 30/32	medium · avg 55%	▇ 0.3%	blunt · at its level · small run
RealWorldQA lab	85.3% ✎	▂▃▂█▃██▆█▂▆▆ beats 36/38	easy · avg 72%	▇ 0.2%	blunt · small run
CC-OCR lab	81.9% ✎	▁·▂··▂▂▃▃▂▆█ beats 24/26	easy · avg 72%	▇ 0.2%	blunt · small run
OCRBench lab	90.0% ✎	▁·▃▁▆▅▂█▅█▁▃ beats 32/37	easy · avg 83%	▇ 0.2%	blunt · small run
AI2D test lab	92.7% ✎	▁▁▅▇▃▇·▄·█·▃ beats 22/28	easy · avg 85%	▇ 0.1%	blunt · small run
PureDocBench document parsing (Avg3 over Clean/Digital/Real)	67.1% ✎	▂·▁·▂▂▅█·▁▂▂ beats 8/22	medium · avg 62%	▇ 0.0%	blunt · at its level

web search▂▁▃▆▄█▃▃▁▁·▁ #72 of 80 * · 1.8% of its score · 2 tests

Trails GPT-4.1 nano, GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
K-BrowseComp-Verified (Korean web-browsing agent questions, pass@1)	12.0% ✎	▃····█··▃█·▃ beats 3/8	very hard · avg 3.4%	▇ 1.2%	typical
WideSearch lab	60.1% ✎	█·▂▆··▂···▂▂ beats 2/10	hard · avg 34%	▇ 0.6%	at its level · small run

other tests29 not in a task · 30% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond +1	83.8% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 142/208	hard · avg 45%	▇▇▇ 6.3%	sharp · several runs
Humanity's Last Exam lab +1	21.4% ✎	▁▁▁▃▆▆▇██▆▁▃ beats 50/104	very hard · avg 5.1%	▇ 2.7%	several runs
MMLU-Pro lab +1	85.2% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 173/188	medium · avg 56%	▇ 2.5%	several runs
NL2Repo lab +1	29.4% ✎	▄·····█·▄▄·▄ beats 2/5	very hard · avg 8.2%	▇ 2.1%	at its level · several runs
QwenClawBench (Qwen internal) lab +1	52.6% ✎	▄·▄····▄···█ beats 4/4	hard · avg 22%	▇ 1.6%	at its level · several runs
Claw-Eval Pass^3 lab +1	50.0% ✎	█▄▄·▄·▄▄·▄·▄ beats 2/8	hard · avg 24%	▇ 1.2%	at its level · several runs
SkillsBench 78-task subset (OpenCode, avg of 5) lab +1	28.7% ✎	▄·····▄···▄█ beats 4/4	very hard · avg 9.9%	▇ 1.0%	at its level · several runs
Mystery Game Puzzles (Epoch)	22.0% ✎	█··▂▃▇▄█▂▃▁▁ beats 39/74	very hard · avg 9.4%	▇ 0.9%	sharp · too hard for it · noisy

21 more tests

test	result	vs the others in its run	difficulty	counts	why
C-Eval lab	90.0% ✎	▁···▁▁▂▁▂▂▃█ beats 22/33	medium · avg 69%	▇ 0.9%	small run
Chess Puzzles (Epoch)	26.0% ✎	█▁▁▁▁▂▁▄▂▂▁▁ beats 97/135	very hard · avg 5.9%	▇ 0.9%	sharp · noisy
VitaBench lab	35.6% ✎	▄··▂▄▆▆█▄▄▄▂ beats 18/21	very hard · avg 11%	▇ 0.9%	at its level · small run
Claw-Eval average score lab +1	68.7% ✎	▄·▄··█▄▄▄▄▄▄ beats 7/9	medium · avg 47%	▇ 0.8%	blunt · at its level · several runs
MMLU-Redux lab	93.3% ✎	▁▁▃▁▂▃▄▅▇▅█▄ beats 54/63	easy · avg 78%	▇ 0.8%	small run
RobloxQA v2.0 multiple choice (Torpedo Software)	89.8% ✎	▁▁▃▁█▄▃▃▆▂▂▂ beats 13/49	easy · avg 78%	▇ 0.7%	typical
LVBench lab	71.4% ✎	▇·█▅▅▄▄··▃▁▁ beats 20/28	medium · avg 52%	▇ 0.6%	at its level · small run
DeepPlanning lab	25.9% ✎	▂·····▂··▄█▄ beats 7/9	very hard · avg 9.0%	▇ 0.6%	small run
ZEROBench subquestions lab	34.4% ✎	▄·▄···▄·▄██▄ beats 5/8	hard · avg 16%	▇ 0.5%	at its level · small run
SuperGPQA lab	64.7% ✎	▂▁▂▂▅▆▃▇▇▇█▃ beats 40/46	medium · avg 36%	▇ 0.5%	at its level · small run
VideoMMMU lab	83.7% ✎	▂···▂···▂▂█▆ beats 8/10	medium · avg 61%	▇ 0.5%	small run
VLMs Are Blind lab	96.6% ✎	▂·▂·····▃··█ beats 4/8	very easy · avg 88%	▇ 0.5%	maxed out · small run
Video-MME (with subtitles) lab	86.6% ✎	▁▁▁▁▁·▁▃▁▁▁█ beats 11/16	medium · avg 71%	▇ 0.5%	small run
ERQA lab	61.8% ✎	▃··▃··█·▃█▅▅ beats 6/12	medium · avg 36%	▇ 0.5%	at its level · small run
MLVU lab	86.2% ✎	▂···▂···▂▄▄█ beats 7/10	easy · avg 69%	▇ 0.4%	small run
RefSpatial-Bench lab	64.3% ✎	▃·▃····▃▃▅▅█ beats 5/10	medium · avg 40%	▇ 0.4%	at its level · small run
DiagnosticIQ industrial maintenance MCQ (macro accuracy, 4 options)	69.9% ✎	▁··▃▁▁▁▅█▅▃▅ beats 20/24	medium · avg 52%	▇ 0.4%	blunt · at its level
EmbSpatial-Bench lab	84.3% ✎	▁·····▁·▁·▃█ beats 6/10	easy · avg 67%	▇ 0.4%	small run
Video-MME (without subtitles) lab	82.5% ✎	▁▃▁▃▁·█▄▄▅·▇ beats 23/27	medium · avg 67%	▇ 0.4%	blunt · small run
NTCIR SciClaimEval Subtask 1 pair-accuracy (dev, single model)	66.2% ✎	▃▃···▃·▃▃█·▃ beats 3/8	medium · avg 45%	▇ 0.3%	blunt · at its level
MVBench lab	74.6% ✎	▁··▃▄▁▃▄▄▄█▄ beats 19/26	medium · avg 66%	▇ 0.2%	blunt · at its level · small run

$ where to run it

Through an API

api · Deep Infra · $0.31/M · $0.1 in / $0.95 out per 1M tokens · 262k context · deepinfra.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3.6 35B-A3B · needs ~20.4 GB · from huggingface.co/ggml-org
chat in the terminal
llama cli -hf ggml-org/Qwen3.6-35B-A3B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf ggml-org/Qwen3.6-35B-A3B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3.6 35B-A3B · needs ~20.4 GB · from huggingface.co/ggml-org
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/ggml-org/Qwen3.6-35B-A3B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3.6 35B-A3B · needs ~21.8 GB · from ollama.com
chat in the terminal
ollama run qwen3.6:35b-a3b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3.6 35B-A3B · needs ~20.4 GB, Mac only · from huggingface.co/lmstudio-community
chat in the terminal
mlx_lm.chat --model lmstudio-community/Qwen3.6-35B-A3B-MLX-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Qwen3.6-35B-A3B-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3.6-27B model card 36%, Qwen/Qwen3.6-35B-A3B model ca… 23%, Epoch AI — internal evaluatio… 15% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Qwen/Qwen3.6-27B model card · lab-reported	41	▇▇▇▇▇▇▇▇▇▇▇ 36%	apache-2.0
Qwen/Qwen3.6-35B-A3B model card · lab-reported	27	▇▇▇▇▇▇▇ 23%	apache-2.0
Epoch AI — internal evaluations	5	▇▇▇▇ 15%	CC BY 4.0
MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets	4	▇▇▇▇ 12%	CC BY-SA 4.0
Kwaipilot/KAT-Coder-V2.5-Dev model card	5	▇▇▇ 9%	Apache-2.0
Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode	2	▇ 3%	MIT
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts (arXiv:2606.02404v1)	1	▇ 1%	CC BY 4.0
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)	1	▇ 0%	CC BY 4.0
SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task — authors' own single-model dev evaluation	1	▇ 0%	CC BY 4.0
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)	1	▇ 0%	CC BY 4.0
prior · pull toward the middle		▇ 0%
weighted average		= 67.7

$ published speed measurements · 16

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_0	81.3 tok/s	llama.cpp 2496f9c14 (build_number 9049 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=2048; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 56 (2026-05-07, measured-local): Fastest measured Qwen3.6 speed-first quant; lower-quality Q4_0, not default all-round quality recommendation.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	76.94 tok/s	llama.cpp 2496f9c14 (build_number 9049 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=1024; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 57 (2026-05-07, measured-local): Fast balanced Strix quant candidate; quality sanity recommended before replacing older UD default row.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	UD-Q4_K_M	62.56 tok/s	llama.cpp 2496f9c14 (build_number 9049 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r20 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 62 (2026-05-07, measured-local): Clean latest-stack r20 rerun; one lightweight workstation dependency remained running; no material change versus b9010 headline.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	UD-Q4_K_M	63.06 tok/s	llama.cpp d05fe1d7d (build_number 9010 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r20 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 68 (2026-05-03, measured-local): Controlled two-run r20 average; current UD penalty not reproduced	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	UD-Q4_K_M	52.69 tok/s	llama.cpp b1c70e2e5 (build_number 8460 as recorded), ROCm/HIP · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r10 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics. Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 96 (2026-05-03, measured-local): ROCm spot check with HSA override; rocBLASLt Tensile gfx1151 file missing warning	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	72.55 tok/s	Ollama 0.31.1 controlled local binary, Ollama Vulkan RADV · Ollama API generation rate (eval tokens/s) as tabulated; prompt 25 tokens; context window setting 2048; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-07-16, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.1.4. Author note: One cold plus nine warm local-server requests with identical cache, environment, prompt, and port shape across versions; warm mean. Separate from the normal system-service result.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	73.19 tok/s	Ollama 0.31.2 controlled local binary, Ollama Vulkan RADV · Ollama API generation rate (eval tokens/s) as tabulated; prompt 25 tokens; context window setting 2048; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-07-16, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.1.4. Author note: Controlled same-host local-server comparator; warm mean over nine requests. Shows the earlier 60.57 t/s installed-service row is not sufficient evidence of a version-wide 0.31.2 regression.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	73.2 tok/s	Ollama 0.32.0 controlled local binary, Ollama Vulkan RADV · Ollama API generation rate (eval tokens/s) as tabulated; prompt 25 tokens; context window setting 2048; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-07-16, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.1.4. Author note: Controlled local-binary compatibility pass; 42/42 Qwen layers offloaded, Qwen2.5-VL vision worked before and after process restart. No system-package upgrade or full-host reboot claim.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	50.51 tok/s	Ollama 0.23.1, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 19 tokens; context window setting 65536; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-05-07, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.0.6. Author note: Controlled API warm average across 10 runs after one cold run; matches 0.21.2 baseline.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	49.05 tok/s	Ollama 0.24.0, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 25 tokens; context window setting 65536; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-05-16, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.0.6. Author note: Same-prompt isolated binary check showed no speedup versus local Ollama 0.23.1 control at 49.09 t/s.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	49.09 tok/s	Ollama 0.23.1, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 25 tokens; context window setting 65536; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-05-16, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.0.6. Author note: Same-prompt control for the isolated Ollama 0.24.0 check.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	50.51 tok/s	Ollama 0.21.2, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 19 tokens; context window setting 65536; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-05-03, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.0.6. Author note: Controlled API warm average across 10 runs	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	63.76 tok/s	llama.cpp b8460, Vulkan (RADV) · llama-bench pp512/tg128 as tabulated (tg_tps column = tg128 generation); exact flags not retained in a raw CSV for this historical row Historical row sourced by the guide to its own BENCHMARKS.md/README.md; no raw CSV retained. Row date 2026-04-26, status measured-local, kernel 6.19.4, driver stack 26.0.2-26.0.6. Author note: Recommended all-rounder	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	60.57 tok/s	Ollama 0.31.2 system service, Ollama Vulkan RADV · Ollama API generation rate (eval tokens/s) as tabulated; prompt 26 tokens; context window setting 2048; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-07-10, status measured-local, kernel 6.19.4-061904-generic, driver stack Mesa 26.1.4 kisak. Author note: Normal installed-service path with OLLAMA_IGPU_ENABLE=1 and existing OLLAMA_NUM_PARALLEL=2; 9 warm API runs after one cold run; 60.00-61.22 t/s range; 100% GPU reported; Qwen2.5-VL 7B vision plus service-restart and full-host-reboot persistence passed; slower than the separate 0.31.1 local-binary run.	source ↗
Apple M5 Pro (48GB; CPU/GPU core counts not stated)	Q4 GGUF (4-bit; exact llama.cpp type not stated)	63.1 tok/s	llama.cpp b9960, Metal backend, default settings · tg128: token-generation throughput over 128 generated tokens; average of five repetitions; macOS (Darwin 25.4) Baseline runtime column from the BaseRT M5 paper; BaseRT's own column is not recorded.	source ↗
Apple M5 Pro (48GB; CPU/GPU core counts not stated)	MLX 4-bit (group size not stated)	101.9 tok/s	mlx-lm 0.31.3 (MLX 0.32.0), default generation settings · tg128: token-generation throughput over 128 generated tokens; average of five repetitions; macOS (Darwin 25.4) Baseline runtime column from the BaseRT M5 paper; BaseRT's own column is not recorded.	source ↗

$ quantization evidence · 21 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	GPQA Diamond 0-shot (lighteval via litellm; mean of 3 seeds)	0.845 → 0.859	101.6%	source ↗
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	GSM8K-Platinum 0-shot (lm-evaluation-harness gsm8k_platinum_cot_llama, chat completions; mean of 3 seeds)	0.957 → 0.96	100.3%	source ↗
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	AIME 2025 0-shot (lighteval via litellm; mean of 8 seeds)	0.929 → 0.929	100.0%	source ↗
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	MATH-500 0-shot (lighteval via litellm; mean of 3 seeds)	0.848 → 0.848	100.0%	source ↗
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	MMLU-Pro 0-shot (lm-evaluation-harness mmlu_pro_chat, chat completions; mean of 3 seeds)	0.853 → 0.853	100.0%	source ↗
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	IFEval 0-shot (lm-evaluation-harness ifeval, chat completions; reported metric variant not stated; mean of 3 seeds)	0.931 → 0.927	99.6%	source ↗
FP8 block-wise (E4M3, 128x128 weight blocks, dynamic activations; config.json quant_method fp8; embeddings, lm_head, norms, MoE router/shared-expert gates, parts of linear attention, vision encoder and MTP excluded per modules_to_not_convert)	LiveCodeBench v6 code generation 0-shot (lighteval via litellm; mean of 3 seeds)	0.773 → 0.766	99.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	LiveCodeBench v6 code generation 0-shot (lighteval via litellm; mean of 3 seeds)	0.773 → 0.789	102.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	GPQA Diamond 0-shot (lighteval via litellm; mean of 3 seeds)	0.845 → 0.849	100.4%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	MMLU-Pro 0-shot (lm-evaluation-harness mmlu_pro_chat, chat completions; mean of 3 seeds)	0.853 → 0.854	100.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	MATH-500 0-shot (lighteval via litellm; mean of 3 seeds)	0.848 → 0.847	99.9%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	GSM8K-Platinum 0-shot (lm-evaluation-harness gsm8k_platinum_cot_llama, chat completions; mean of 3 seeds)	0.957 → 0.956	99.9%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	IFEval 0-shot (lm-evaluation-harness ifeval, chat completions; reported metric variant not stated; mean of 3 seeds)	0.931 → 0.927	99.6%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet), vision-encoder and MTP layers unquantized	AIME 2025 0-shot (lighteval via litellm; mean of 8 seeds)	0.929 → 0.925	99.5%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	GSM8K-Platinum 0-shot (lm-evaluation-harness gsm8k_platinum_cot_llama, chat completions; mean of 3 seeds)	0.957 → 0.961	100.4%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	MATH-500 0-shot (lighteval via litellm; mean of 3 seeds)	0.848 → 0.85	100.2%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	GPQA Diamond 0-shot (lighteval via litellm; mean of 3 seeds)	0.845 → 0.847	100.2%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	IFEval 0-shot (lm-evaluation-harness ifeval, chat completions; reported metric variant not stated; mean of 3 seeds)	0.931 → 0.924	99.3%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	MMLU-Pro 0-shot (lm-evaluation-harness mmlu_pro_chat, chat completions; mean of 3 seeds)	0.853 → 0.847	99.3%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	AIME 2025 0-shot (lighteval via litellm; mean of 8 seeds)	0.929 → 0.912	98.2%	source ↗
NVFP4 W4A4 (FP4 E2M1 weights and activations, group 16; llm-compressor QuantizationModifier NVFP4, compressed-tensors); lm_head, embeddings, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision layers unquantized	LiveCodeBench v6 code generation 0-shot (lighteval via litellm; mean of 3 seeds)	0.773 → 0.747	96.6%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json