whatmodel.sh / Qwen3.5 35B-A3B

Qwen3.5 35B-A3B

+ contribute

Qwen3.5 35B-A3B is an open-weights language model from Alibaba, released February 2026. Its published size is 36B parameters (3B active per token) and a 262k-token context window. Licence: apache-2.0.

66.6±0.4#120 of 719 LLMsfrom 135 benchmark results across 18 sources

$ how it compares

The LLMs ranked either side of Qwen3.5 35B-A3B, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
137	Gemini 2.5 Pro Preview (05-06) Google	67±1	comparison only		2025-05
138	GigaChat 3.5 Ultra Reasoning (432B-A28B) Sber	67±1	local	~461GB	2026-09
139	Qwen3.5 35B-A3B Alibaba	67±0	api $0.35/M +local	~23GB	2026-02
140	DeepSeek V3.1 Terminus DeepSeek	67±1	local	~230GB	2025-09
141	DeepSeek V3.2 Exp DeepSeek	67±1	local	~726GB	2025-09
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #95 of 300 · 22% of its score · 18 tests

Beats Grok 4.20 (0309 reasoning), Grok 4.3, o3-pro here, though they rank higher overall. Trails Nova 2 Pro (Preview) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified +4	58.6% ✎	▄▄·····▄▄▄▄█ beats 3/7	hard · avg 17%	▇▇ 6.1%	sharp · at its level · several runs
Terminal-Bench 2.1	26.4% ✎	▄······█▄▄█▄ beats 1/7	very hard · avg 7.1%	▇ 2.0%	at its level
τ²-bench aggregate lab +1	81.2% ✎	▁▁▁·▂▂▄▃▃▅█▂ beats 25/37	medium · avg 36%	▇ 1.9%	several runs
SWE-bench Multilingual +1	47.7% ✎	▄·▄·····█▄█▄ beats 2/7	hard · avg 18%	▇ 1.8%	at its level · several runs
SWE-bench Pro (split unspecified)	38.0% ✎	▄···▄···▄▄██ beats 4/7	very hard · avg 6.5%	▇ 1.6%	sharp · at its level
Terminal-Bench 2.0 lab	40.5% ✎	▂▂▅▅████▆▂▃▃ beats 13/36	very hard · avg 11%	▇ 1.1%	at its level · small run
WideSearch lab +1	57.1% ✎	█·▂▆··▂···▂▂ beats 1/10	hard · avg 33%	▇ 1.0%	at its level · small run · several runs
BrowseComp-ZH lab	69.5% ✎	▁▁▁▁▁▃·▁▄▅▄█ beats 20/23	hard · avg 23%	▇ 0.9%	at its level · small run

10 more tests

test	result	vs the others in its run	difficulty	counts	why
OSWorld-Verified lab	54.5% ✎	█▄·▃▆▂▄▂▆▄▆█ beats 6/25	hard · avg 21%	▇ 0.8%	at its level · small run
BrowseComp lab	61.0% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 24/42	hard · avg 23%	▇ 0.8%	at its level · small run
MCP-Atlas (public set) lab	62.4% ✎	▂·▁▂▁·▁▃█▆▆▂ beats 12/37	hard · avg 29%	▇ 0.7%	at its level · small run
MCPMark lab	27.0% ✎	▃·····█··▃█▃ beats 4/8	very hard · avg 10%	▇ 0.6%	at its level · small run
Toolathlon (Tool Decathlon) lab	28.7% ✎	▂▂▄▃▄▄▆▂█▆██ beats 4/27	very hard · avg 11%	▇ 0.6%	at its level · small run
Humanity's Last Exam with tools lab	47.4% ✎	▂▂▂▁▄▂▃▆█▇▂▆ beats 23/54	hard · avg 18%	▇ 0.6%	at its level · small run
EEGAgentBench overall (tool-based EEG analysis agents, 6 tasks)	45.9% ✎	▂·▁▁··▇▇█▂·▁ beats 11/25	hard · avg 29%	▇ 0.5%	blunt · at its level
BFCL V4 (December 2025 archive) lab	67.3% ✎	▁▁▁▂▇▆▆▆█▂▄▂ beats 100/111	medium · avg 41%	▇ 0.4%	at its level · small run
SWE-bench Pro public set, Qwen-refined lab	44.6% ✎	▂·····▂·▃█▃▅ beats 2/12	hard · avg 15%	▇ 0.4%	at its level
τ³-bench lab	68.9% ✎	▁▁▂▁·▁▁·▁▁▂█ beats 17/25	medium · avg 38%	▇ 0.4%	blunt · at its level · small run

coding▁▁▁▃▆██▇▆▄▃▁ #140 of 458 · 18% of its score · 9 tests

Beats Gemini 2.5 Pro, Gemini 3.1 Flash-Lite, Qwen3-Coder-Next (80B-A3B) here, though they rank higher overall. Trails Devstral Small 2 24B, Llama 3.1 Nemotron Ultra 253B v1, Nemotron 3.5 Lightning 30B-A3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified +4	58.6% ✎	▄▄·····▄▄▄▄█ beats 3/7	hard · avg 17%	▇▇ 6.1%	sharp · at its level · several runs
Arena WebDev (Code Arena)	1253.4 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 13/119	avg 981	▇ 2.9%	sharp
LiveCodeBench v6 (Feb–May 2025) lab +3	74.6% ✎	▁▁▃▁█▅▄▇▅▅▃█ beats 24/38	medium · avg 43%	▇ 2.4%	at its level · several runs
SWE-bench Multilingual +1	47.7% ✎	▄·▄·····█▄█▄ beats 2/7	hard · avg 18%	▇ 1.8%	at its level · several runs
SWE-bench Pro (split unspecified)	38.0% ✎	▄···▄···▄▄██ beats 4/7	very hard · avg 6.5%	▇ 1.6%	sharp · at its level
LiveCodeBench Pro 2025Q2 (Medium)	17.8% ✎	█··▄▂▂██▄·▄▆ beats 19/22	very hard · avg 3.0%	▇ 1.6%	sharp
SciCode (level unspecified)	27.7% ✎	▄···▄▄█▄·▄·▄ beats 2/7	very hard · avg 13%	▇ 0.8%	at its level
LiveCodeBench Pro 2025Q2 (Easy)	81.1% ✎	▂··▃▂▃▆▅█·▃▂ beats 18/20	medium · avg 45%	▇ 0.7%	typical

1 more test

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Pro public set, Qwen-refined lab	44.6% ✎	▂·····▂·▃█▃▅ beats 2/12	hard · avg 15%	▇ 0.4%	at its level

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #119 of 393 · 2.7% of its score · 3 tests

Beats LongCat 2.0 (1.6T-A48B), GPT-5 mini here, though they rank higher overall. Trails LFM2.5 2.6B, Nemotron-Cascade-8B, DeepSeek V3 0324 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1393.8 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 236/379	avg 1268	▇ 1.6%	blunt
Arena-Hard v2 aggregate	65.4% ✎	▁▁··▄▃▁▃█▁▁▇ beats 13/22	hard · avg 28%	▇ 0.7%	sharp · at its level · small run
IFEval (aggregation unspecified) lab	91.9% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 122/141	easy · avg 81%	▇ 0.4%	small run

math▁▂▃▅▆▅▄█▄▁▁▁ #127 of 398 · 18% of its score · 14 tests

Beats Claude Sonnet 4.6, MiniMax-M3, MiMo-V2.5 (310B-A15B) here, though they rank higher overall. Trails Olmo 3.1 32B Think, Qwen3-VL 30B-A3B Thinking, Granite 4.2 8B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
AIME 2026 +1	91.1% ✎	▁▁▁▂▁▂▆█▆▂·▁ beats 25/49	medium · avg 57%	▇ 2.5%	sharp · small run · several runs
IMO-AnswerBench +1	74.8% ✎	▁▁·▁▁▁▄▅█▄▂▃ beats 16/42	medium · avg 36%	▇ 2.2%	small run · several runs
MathArena ArXivMath February 2026 (32 problems)	30.5% ✎	▁▁▁▁█▇▄▃▁▃·▃ beats 4/24	very hard · avg 9.1%	▇ 1.8%	at its level
HMMT February 2026 lab	78.7% ✎	▃▃▂▃·██▆█▅▃▅ beats 18/33	hard · avg 35%	▇ 1.6%	sharp · small run
MathArena ArXivMath January 2026 (23 problems)	50.0% ✎	▂··▄▂▄█▆▆▄▄█ beats 5/23	hard · avg 22%	▇ 1.5%	at its level
MATH-Vision lab +1	83.9% ✎	▁▁·▁··▄▁▃▇▇█ beats 18/24	medium · avg 54%	▇ 1.4%	small run · several runs
OTIS mock AIME 2024–2025	54.4% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 81/193	very hard · avg 8.5%	▇ 1.2%	sharp · noisy
HMMT February 2025 lab	89.0% ✎	▁▁▁▃▁▅▅▃██▁▄ beats 16/31	medium · avg 61%	▇ 1.2%	small run

6 more tests

test	result	vs the others in its run	difficulty	counts	why
AIME 2025	91.9% ✎	▁·▁▁▁▂▅▆█▇▅▁ beats 120/133	medium · avg 63%	▇ 1.0%	sharp · maxed out
MathArena ArXivMath December 2025 (17 problems)	39.7% ✎	▃·▃▃█▁·▄·▁▁▁ beats 7/18	hard · avg 19%	▇ 1.0%	at its level
HMMT November 2025 lab	89.2% ✎	▁▁····▁▅▃█▇▁ beats 12/23	easy · avg 67%	▇ 0.7%	small run
MathVista (testmini) lab	86.2% ✎	▁··▁▄▄▇█▅▆█▄ beats 42/47	medium · avg 64%	▇ 0.7%	small run
PolyMATH lab	64.4% ✎	▁···▃▁·▇█▄▄▄ beats 19/23	hard · avg 34%	▇ 0.6%	at its level · small run
DynaMath lab	85.0% ✎	▂·····▂··█▄▆ beats 6/10	medium · avg 65%	▇ 0.6%	small run

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #51 of 147 · 10% of its score · 19 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5, Claude Opus 4.5 here, though they rank higher overall. Trails Qwen3.6 Flash here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH-Vision lab +1	83.9% ✎	▁▁·▁··▄▁▃▇▇█ beats 18/24	medium · avg 54%	▇ 1.4%	small run · several runs
OmniDocBench 1.5 overall (higher is better) lab +1	89.3% ✎	▂··▂▂▂··▂▂█▅ beats 7/13	easy · avg 70%	▇ 1.1%	small run · several runs
ChartArena chart parsing (English, mAP_high average)	54.1% ✎	▁·▁·▃▃▁▇▃█·▄ beats 20/22	hard · avg 27%	▇ 0.9%	blunt · at its level
CharXiv (reasoning questions) lab	77.5% ✎	▂▅▃▅▅▆▆▅▆█▆▂ beats 69/100	medium · avg 43%	▇ 0.8%	at its level · small run
MathVista (testmini) lab	86.2% ✎	▁··▁▄▄▇█▅▆█▄ beats 42/47	medium · avg 64%	▇ 0.7%	small run
MMMU (val) lab	81.4% ✎	▁····▁▁▄█▇▆▆ beats 47/52	medium · avg 58%	▇ 0.7%	small run
MMMU Pro lab	75.1% ✎	▂▃▁▃▆▇▃▇▇██▅ beats 39/52	medium · avg 47%	▇ 0.6%	at its level · small run
MMStar lab	81.9% ✎	▃▅▂▃▂▂▆█▆▆█▅ beats 32/34	medium · avg 63%	▇ 0.5%	small run

11 more tests

test	result	vs the others in its run	difficulty	counts	why
HallusionBench lab +1	67.9% ✎	▃▁·▃▃▃▄▅▄█▅▅ beats 28/32	medium · avg 55%	▇ 0.4%	blunt · at its level · small run · several runs
CountBench lab	97.8% ✎	▂▂▃▂·▂▂▃▃·▅█ beats 17/18	very easy · avg 92%	▇ 0.4%	maxed out · small run
PMC-VQA lab	62.0% ✎	▃···▃··▃▃·▃█ beats 4/7	medium · avg 39%	▇ 0.4%	at its level · small run
SimpleVQA lab	58.3% ✎	▃·▃·▃·▃·█▅▃▃ beats 6/10	hard · avg 34%	▇ 0.4%	at its level · small run
MMBench-EN dev v1.1 lab	91.5% ✎	█▄···▂··▄▃▄▆ beats 9/14	easy · avg 82%	▇ 0.3%	blunt · small run
ScreenSpot-Pro lab	68.6% ✎	▂▃▅▆██▅▂···▂ beats 17/24	medium · avg 48%	▇ 0.3%	blunt · at its level · small run
RealWorldQA lab	84.1% ✎	▂▃▂█▃██▆█▂▆▆ beats 34/38	easy · avg 72%	▇ 0.2%	blunt · small run
AI2D test lab +1	92.6% ✎	▁▁▅▇▃▇·▄·█·▃ beats 21/28	easy · avg 85%	▇ 0.2%	blunt · small run · several runs
CC-OCR lab	80.7% ✎	▁·▂··▂▂▃▃▂▆█ beats 19/26	easy · avg 72%	▇ 0.2%	blunt · small run
OCRBench lab	91.0% ✎	▁·▃▁▆▅▂█▅█▃▃ beats 34/37	easy · avg 83%	▇ 0.1%	blunt · small run
PureDocBench document parsing (Avg3 over Clean/Digital/Real)	65.7% ✎	▂·▁·▂▃▅█·▁▂▂ beats 6/22	medium · avg 62%	▇ 0.0%	blunt · at its level

web search▂▁▂▆▄█▃▃▁▁·▁ #65 of 80 · 3.7% of its score · 5 tests

Beats DeepSeek V4 Flash (Preview, Apr 2026), Nemotron 3 Ultra 550B-A55B, GLM-4.7 here, though they rank higher overall. Trails GPT-4.1 nano, GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
WideSearch lab +1	57.1% ✎	█·▂▆··▂···▂▂ beats 1/10	hard · avg 33%	▇ 1.0%	at its level · small run · several runs
BrowseComp-ZH lab	69.5% ✎	▁▁▁▁▁▃·▁▄▅▄█ beats 20/23	hard · avg 23%	▇ 0.9%	at its level · small run
BrowseComp lab	61.0% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 24/42	hard · avg 23%	▇ 0.8%	at its level · small run
Humanity's Last Exam with tools lab	47.4% ✎	▂▂▂▁▄▂▃▆█▇▂▆ beats 23/54	hard · avg 18%	▇ 0.6%	at its level · small run
SealQA Seal-0 lab	41.4% ✎	█····█·····█ beats 0/2	hard · avg 19%	▇ 0.5%	at its level · small run

other tests52 not in a task · 41% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond +3	83.5% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 140/208	hard · avg 40%	▇▇▇ 6.9%	sharp · at its level · several runs
Humanity's Last Exam lab +3	22.4% ✎	▁▁▁▃▆▆▇██▆▁▃ beats 52/104	very hard · avg 5.2%	▇ 2.9%	several runs
MMLU-Pro lab +3	85.3% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 174/188	medium · avg 57%	▇ 2.9%	several runs
VitaBench lab +1	31.9% ✎	▄··▂▄▆▆█▄▄▄▂ beats 15/21	very hard · avg 11%	▇ 1.3%	at its level · small run · several runs
MedXPertQA MM lab	61.4% ✎	▂···▂·▃··▆▂█ beats 10/13	very hard · avg 31%	▇ 1.3%	sharp · at its level · small run
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	35.4% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 27/39	very hard · avg 22%	▇ 1.1%	blunt · at its level
Polish medical board exams, bias-mitigated MCQ protocol (7-exam average)	72.2% ✎	▄·▂▂▂··█▂▄·▄ beats 12/13	hard · avg 48%	▇ 1.0%	at its level
LongBench v2 lab	59.0% ✎	▂·▁·▂▂▁▃▄▁█▄ beats 18/28	very hard · avg 35%	▇ 1.0%	at its level · small run

44 more tests

test	result	vs the others in its run	difficulty	counts	why
C-Eval lab	90.2% ✎	▁···▁▁▂▁▂▂▃█ beats 23/33	medium · avg 70%	▇ 0.9%	small run
LVBench lab +1	71.4% ✎	▇·█▅▅▄▄··▃▁▁ beats 21/28	medium · avg 52%	▇ 0.9%	at its level · small run · several runs
AgentWorldBench (open-ended, overall) lab	47.7% ✎	▄▄··▄▄·▄█▄██ beats 1/11	hard · avg 25%	▇ 0.9%	blunt · at its level
INCLUDE lab	79.7% ✎	▂····▂▂▄▄▄█▄ beats 27/36	medium · avg 55%	▇ 0.8%	at its level · small run
DeepPlanning lab +1	22.8% ✎	▂·····▂··▄█▄ beats 5/9	very hard · avg 8.7%	▇ 0.8%	small run · several runs
ZEROBench subquestions lab +1	34.1% ✎	▄·▄···▄·▄██▄ beats 4/8	hard · avg 16%	▇ 0.8%	at its level · small run · several runs
MMLU-Redux lab	93.3% ✎	▁▁▃▁▂▃▄▅▇▅█▄ beats 55/63	easy · avg 78%	▇ 0.8%	small run
NL2Repo lab	20.5% ✎	▂▂·▃▂▄█▆▆▂▄▆ beats 2/21	very hard · avg 6.6%	▇ 0.8%	typical
MMMLU lab	85.2% ✎	▂▁▁▂▃▂▄▄▅██▃ beats 36/46	medium · avg 63%	▇ 0.7%	small run
OJBench lab	36.0% ✎	▁···▁▃·▅▄▅██ beats 21/26	very hard · avg 13%	▇ 0.7%	at its level · small run
QwenClawBench (Qwen internal) lab	47.7% ✎	▄·▄····▄···█ beats 2/4	hard · avg 22%	▇ 0.7%	at its level
Video-MME (without subtitles) lab +1	82.5% ✎	▁▃▁▃▁·█▄▄▅·▇ beats 24/27	medium · avg 67%	▇ 0.6%	blunt · small run · several runs
CPTUB average (SpeakLeash, 0-5 judge scale)	84.5% ✎	▁▂▂▂▃█▃▆▅▄▃▃ beats 89/95	easy · avg 66%	▇ 0.6%	typical
Global PIQA lab	86.6% ✎	▃▃··▃··▃▃▅█▅ beats 8/11	medium · avg 72%	▇ 0.6%	at its level · small run
Chess Puzzles (Epoch)	1.0% ✎	█▁▁▁▁▂▁▄▂▂▁▁ beats 44/135	very hard · avg 5.9%	▇ 0.6%	sharp · noisy
MAXIFE lab	86.6% ✎	▂·▂·▂···▆·▄█ beats 8/11	easy · avg 65%	▇ 0.6%	small run
IFBench (aggregation unspecified) lab	70.2% ✎	▃▇▂▄▅▅▂▄▄▆█▇ beats 55/80	medium · avg 43%	▇ 0.6%	at its level · small run
MultiChallenge lab	60.0% ✎	▁·▁·▁▃▆▅▃▁█▅ beats 19/29	hard · avg 30%	▇ 0.5%	at its level · small run
VideoMMMU lab	80.4% ✎	▂···▂···▂▂█▆ beats 5/10	medium · avg 59%	▇ 0.5%	small run
MMLU-ProX lab	81.0% ✎	▁·▁▁▁▂▁▃▂█▅▄ beats 25/35	medium · avg 57%	▇ 0.5%	small run
SuperGPQA lab	63.4% ✎	▂▁▂▂▅▆▃▇▇▇█▃ beats 38/46	medium · avg 36%	▇ 0.5%	at its level · small run
VLMs Are Blind lab	97.0% ✎	▂·▂·····▃··█ beats 8/8	very easy · avg 89%	▇ 0.5%	maxed out · small run
LingoQA lab	79.2% ✎	▄······▄·█·█ beats 1/5	medium · avg 56%	▇ 0.5%	small run
SLAKE lab	78.7% ✎	▄·····▄▄·██▄ beats 3/7	medium · avg 59%	▇ 0.5%	small run
Video-MME (with subtitles) lab	86.6% ✎	▁▁▁▁▁·▁▃▁▁▁█ beats 12/16	medium · avg 71%	▇ 0.5%	small run
MMVU lab	72.3% ✎	▃···▃···▃▃▅█ beats 5/8	medium · avg 48%	▇ 0.5%	at its level · small run
Claw-Eval Pass^3 lab	51.0% ✎	█▄▄·▄·▄▄·▄·▄ beats 3/8	hard · avg 24%	▇ 0.5%	at its level · small run
MLVU lab	85.6% ✎	▂···▂···▂▄▄█ beats 5/10	easy · avg 69%	▇ 0.4%	small run
RefSpatial-Bench lab	63.5% ✎	▃·▃····▃▃▅▅█ beats 4/10	medium · avg 40%	▇ 0.4%	at its level · small run
FullStackBench (English) lab	58.1% ✎	█····█·····█ beats 0/2	hard · avg 31%	▇ 0.4%	at its level · small run
FullStackBench (Chinese) lab	55.0% ✎	█······█···█ beats 0/2	hard · avg 29%	▇ 0.4%	at its level · small run
ERQA lab	64.8% ✎	▃··▃··█·▃█▅▅ beats 9/12	medium · avg 36%	▇ 0.4%	at its level · small run
MMLongBench-Doc lab	59.5% ✎	▃····▃··▃▃▅█ beats 6/8	medium · avg 35%	▇ 0.4%	at its level · small run
EmbSpatial-Bench lab	83.1% ✎	▁·····▁·▁·▃█ beats 4/10	easy · avg 66%	▇ 0.4%	small run
NOVA-63 lab	57.1% ✎	▃▃·▃···▃█▃▃█ beats 8/11	medium · avg 35%	▇ 0.4%	blunt · at its level · small run
AndroidWorld lab	71.1% ✎	▁·▁▁▂▃█▅▂▃·▂ beats 22/24	medium · avg 41%	▇ 0.4%	blunt · at its level · small run
SkillsBench 78-task subset (OpenCode, avg of 5) lab	4.4% ✎	▄·····▄···▄█ beats 0/4	very hard · avg 7.1%	▇ 0.3%	typical
CL-bench	15.5% ✎	█··········▄ beats 2/2	very hard · avg 7.1%	▇ 0.3%	blunt · small run
Claw-Eval average score lab	65.4% ✎	▄·▄··█▄▄▄▄▄▄ beats 5/9	medium · avg 47%	▇ 0.3%	blunt · at its level · small run
NTCIR SciClaimEval Subtask 1 pair-accuracy (dev, single model)	61.1% ✎	▃▃···▃·▃▃█·▃ beats 2/8	medium · avg 45%	▇ 0.3%	blunt · at its level
MVBench lab	74.8% ✎	▁··▃▄▁▃▄▄▄█▄ beats 21/26	medium · avg 66%	▇ 0.2%	blunt · at its level · small run
LIBRA Mini (Russian long context, May 2026 version)	43.7% ✎	▄···▄█·██··▄ beats 6/8	medium · avg 38%	▇ 0.1%	blunt · at its level
WMT24++ lab	76.3% ✎	▁·▁▁·▁·▁▂▆█▃ beats 7/27	easy · avg 69%	▇ 0.1%	blunt · small run
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	89.5% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 40/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · Deep Infra · $0.35/M · $0.14 in / $1 out per 1M tokens · 262k context · deepinfra.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3.5 35B-A3B · needs ~21.2 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Qwen3.5-35B-A3B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Qwen3.5-35B-A3B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3.5 35B-A3B · needs ~21.2 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Qwen3.5-35B-A3B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3.5 35B-A3B · needs ~22.1 GB · from ollama.com
chat in the terminal
ollama run qwen3.5:35b-a3b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3.5 35B-A3B · needs ~20.4 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Qwen3.5-35B-A3B-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Qwen3.5-35B-A3B-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3.5-122B-A10B model… 42%, Qwen/Qwen3.6-35B-A3B model ca… 18%, Qwen/Qwen3.5-27B model card 8% and 15 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Qwen/Qwen3.5-122B-A10B model card · lab-reported	71	▇▇▇▇▇▇▇▇▇▇▇▇▇ 42%	apache-2.0
Qwen/Qwen3.6-35B-A3B model card · lab-reported	27	▇▇▇▇▇ 18%	apache-2.0
Qwen/Qwen3.5-27B model card · lab-reported	7	▇▇▇ 8%	apache-2.0
Kwaipilot/KAT-Coder-V2.5-Dev model card	5	▇▇ 7%	Apache-2.0
NVIDIA — Nemotron-Cascade 2 technical report	7	▇▇ 6%	CC BY 4.0
LMArena leaderboard dataset	2	▇ 4%	CC BY 4.0
MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets	3	▇ 4%	CC BY-SA 4.0
Epoch AI — internal evaluations	3	▇ 4%	CC BY 4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 1%	CC BY 4.0
Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)	1	▇ 1%	CC BY-SA 4.0
ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)	1	▇ 1%	CC BY 4.0
Qwen/Qwen-AgentWorld-35B-A3B model card · lab-reported	1	▇ 1%	Apache-2.0
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 1%	MIT
EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)	1	▇ 0%	CC BY 4.0
SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task — authors' own single-model dev evaluation	1	▇ 0%	CC BY 4.0
ai-forever LIBRA leaderboard (Russian long-context benchmark)	1	▇ 0%	MIT
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)	1	▇ 0%	CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 66.6

not counted (5)

Qwen/Qwen3.5-35B-A3B model card	MMLU-Pro [correct]	85.3%	the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
Qwen/Qwen3.5-35B-A3B model card	Humanity's Last Exam [correct]	22.4%	the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
Qwen/Qwen3.5-35B-A3B model card	GPQA Diamond [correct]	84.2%	the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
Qwen/Qwen3.5-35B-A3B model card	SWE-bench Verified [correct]	69.2%	the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
Qwen/Qwen3.5-35B-A3B model card	LiveCodeBench v6 (Feb–May 2025) [correct]	74.6%	the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts

$ published speed measurements · 11

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	IQ4_XS	75.22 tok/s	llama.cpp 1fd5f4803 (build_number 134 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r5 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 42 (2026-06-03, measured-local): Current/larger Qwen comparator; newer or larger is not automatically faster. Generation-only p0/n128 r10 was 75.53 t/s.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	64.85 tok/s	llama.cpp b8460, Vulkan (RADV) · llama-bench pp512/tg128 as tabulated (tg_tps column = tg128 generation); exact flags not retained in a raw CSV for this historical row Historical row sourced by the guide to its own BENCHMARKS.md/README.md; no raw CSV retained. Row date 2026-03-21, status measured-local, kernel 6.19.4, driver stack 26.0.2-26.0.6. Author note: Backend and build comparison baseline	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	48 tok/s	Ollama 0.20.4, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 14 tokens; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-03-20, status historical-local, kernel 6.19.4, driver stack 26.0.2. Author note: Mesa 26.0.2 era	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	47.5 tok/s	Ollama 0.20.4, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 23 tokens; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-03-20, status historical-local, kernel 6.19.4, driver stack 26.0.2. Author note: Mesa 26.0.2 era	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	47.4 tok/s	Ollama 0.20.4, Ollama Vulkan RADV · Ollama generation rate (eval tokens/s) as tabulated; prompt 122 tokens; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-03-20, status historical-local, kernel 6.19.4, driver stack 26.0.2. Author note: Mesa 26.0.2 era	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	52.06 tok/s	llama.cpp b8298, Vulkan (RADV) · llama-bench pp512/tg128 as tabulated (tg_tps column = tg128 generation); exact flags not retained in a raw CSV for this historical row Historical row sourced by the guide to its own BENCHMARKS.md/README.md; no raw CSV retained. Row date 2026-03-21, status historical-local, kernel 6.19.4, driver stack 26.0.2-26.0.6. Author note: kyuz0-era direct path	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	54.67 tok/s	llama.cpp b8460, ROCm (HIP) · llama-bench pp512/tg128 as tabulated (tg_tps column = tg128 generation); exact flags not retained in a raw CSV for this historical row Historical row sourced by the guide to its own BENCHMARKS.md/README.md; no raw CSV retained. Row date 2026-03-21, status measured-local, kernel 6.19.4, driver stack 26.0.2-26.0.6. Author note: HSA override fix applied	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	47.87 tok/s	llama.cpp b8301, ROCm (HIP) · llama-bench pp512/tg128 as tabulated (tg_tps column = tg128 generation); exact flags not retained in a raw CSV for this historical row Historical row sourced by the guide to its own BENCHMARKS.md/README.md; no raw CSV retained. Row date 2026-03-21, status historical-local, kernel 6.19.4, driver stack 26.0.2-26.0.6. Author note: Old self-compiled ROCm build with HSA fix	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	48.8 tok/s	llama.cpp b8301, ROCm (HIP) · llama-bench pp512/tg128 as tabulated (tg_tps column = tg128 generation); exact flags not retained in a raw CSV for this historical row Historical row sourced by the guide to its own BENCHMARKS.md/README.md; no raw CSV retained. Row date 2026-03-21, status historical-local, kernel 6.18.14, driver stack 26.0.2-26.0.6. Author note: Previous kernel reference	source ↗
Apple M5 Pro (48GB; CPU/GPU core counts not stated)	Q4 GGUF (4-bit; exact llama.cpp type not stated)	72.5 tok/s	llama.cpp b9960, Metal backend, default settings · tg128: token-generation throughput over 128 generated tokens; average of five repetitions; macOS (Darwin 25.4) Baseline runtime column from the BaseRT M5 paper; BaseRT's own column is not recorded.	source ↗
Apple M5 Pro (48GB; CPU/GPU core counts not stated)	MLX 4-bit (group size not stated)	104.8 tok/s	mlx-lm 0.31.3 (MLX 0.32.0), default generation settings · tg128: token-generation throughput over 128 generated tokens; average of five repetitions; macOS (Darwin 25.4) Baseline runtime column from the BaseRT M5 paper; BaseRT's own column is not recorded.	source ↗

$ quantization evidence · 7 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	LiveCodeBench v6 code generation 0-shot (lighteval via litellm; mean of 3 seeds)	0.754 → 0.764	101.3%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	GPQA Diamond 0-shot (lighteval via litellm; mean of 3 seeds)	0.83 → 0.832	100.2%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	GSM8K-Platinum 0-shot (lm-evaluation-harness gsm8k_platinum_cot_llama, chat completions; mean of 3 seeds)	0.95 → 0.951	100.1%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	MMLU-Pro 0-shot (lm-evaluation-harness mmlu_pro_chat, chat completions; mean of 3 seeds)	0.857 → 0.857	100.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	AIME 2025 0-shot (lighteval via litellm; mean of 8 seeds)	0.912 → 0.912	100.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	IFEval prompt-level strict 0-shot (lm-evaluation-harness ifeval, chat completions; mean of 3 seeds)	0.91 → 0.904	99.4%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	MATH-500 0-shot (lighteval via litellm; mean of 3 seeds)	0.849 → 0.843	99.3%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json