whatmodel.sh / Qwen3.5 122B-A10B

Qwen3.5 122B-A10B

+ contribute

Qwen3.5 122B-A10B is an open-weights language model from Alibaba, released February 2026. Its published size is 125B parameters (10B active per token) and a 262k-token context window. Licence: apache-2.0.

68.8±0.6#96 of 719 LLMsfrom 89 benchmark results across 11 sources

$ how it compares

The LLMs ranked either side of Qwen3.5 122B-A10B, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
103	MiMo-V2.5 (310B-A15B) Xiaomi	69±1	api $0.21/M +local	~100GB	2026-04
104	Qwen3.5 Plus Alibaba	69±2	api $0.90/M		2026-02
105	Qwen3.5 122B-A10B Alibaba	69±1	api $1.10/M +local	~75GB	2026-02
106	Gemini 3.5 Flash-Lite Google	69±1	api $0.85/M		2026-07
107	GLM-4.7 Z.ai	69±1	api $1.00/M +local	~58GB	2025-12
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #86 of 300 · 17% of its score · 10 tests

Beats Grok 4.20 (0309 reasoning), GPT-5.1, Grok 4.3 here, though they rank higher overall. Trails Hy3 preview (295B-A21B), MiniMax-M2.7, Step 3.5 Flash (196B-A11B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified lab +1	72.0% ✎	▁▁▁▁▁▁▂▄█▆▁▁ beats 84/131	hard · avg 24%	▇▇ 4.5%	sharp · at its level · small run · several runs
τ²-bench aggregate lab +1	79.5% ✎	▁▁▁·▂▂▄▃▃▅█▂ beats 23/37	medium · avg 36%	▇ 3.0%	several runs
Terminal-Bench 2.0 lab	49.4% ✎	▂▂▅▅████▆▂▃▃ beats 19/36	very hard · avg 11%	▇ 2.0%	at its level · small run
BrowseComp-ZH lab	69.9% ✎	▁▁▁▁▁▃·▁▄▅▄█ beats 21/23	hard · avg 23%	▇ 1.6%	at its level · small run
OSWorld-Verified lab	58.0% ✎	█▄·▂▆▂▄▂▆▄▆█ beats 8/25	hard · avg 21%	▇ 1.4%	at its level · small run
BrowseComp lab	63.8% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 29/42	hard · avg 23%	▇ 1.3%	at its level · small run
Humanity's Last Exam with tools lab	47.5% ✎	▂▂▂▁▄▂▃▆█▇▂▆ beats 24/54	hard · avg 18%	▇ 1.0%	at its level · small run
WideSearch lab	60.5% ✎	█·▂▆··▂···▂▂ beats 3/10	hard · avg 33%	▇ 0.8%	at its level · small run

2 more tests

test	result	vs the others in its run	difficulty	counts	why
EEGAgentBench overall (tool-based EEG analysis agents, 6 tasks)	48.5% ✎	▂·▁▁··▇▇█▂·▁ beats 15/25	hard · avg 29%	▇ 0.8%	blunt · at its level
BFCL V4 (December 2025 archive) lab	72.2% ✎	▁▁▁▂▇▆▆▆█▂▄▃ beats 106/111	medium · avg 41%	▇ 0.8%	at its level · small run

coding▁▁▁▃▆██▇▆▄▃▁ #104 of 458 · 12% of its score · 3 tests

Beats Grok 4.3, Muse Glimmer 30B, Qwen3.5 Plus here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Solar Pro 4, Nemotron Cascade 2 30B-A3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena WebDev (Code Arena)	1359.8 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 33/119	avg 981	▇▇ 5.0%	sharp
SWE-bench Verified lab +1	72.0% ✎	▁▁▁▁▁▁▂▄█▆▁▁ beats 84/131	hard · avg 24%	▇▇ 4.5%	sharp · at its level · small run · several runs
LiveCodeBench v6 (Feb–May 2025) lab +1	78.9% ✎	▁▁▃▁█▅▄▇▅▅▃█ beats 29/38	medium · avg 43%	▇ 2.7%	several runs

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #92 of 393 * · 3.4% of its score · 2 tests

Beats LongCat 2.0 (1.6T-A48B), Inkling Small (266B-A12B) here, though they rank higher overall. Trails Gemma 4 12B, Kimi K2 Instruct (0905), Gemma 4 26B-A4B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1416.1 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 266/379	avg 1268	▇ 2.8%	blunt
IFEval (aggregation unspecified) lab	93.4% ✎	▁▁▃▁▂▄▄▆█▃▃▁ beats 133/141	easy · avg 81%	▇ 0.6%	maxed out · small run

math▁▂▃▅▆▅▄█▄▁▁▁ #76 of 398 · 8.2% of its score · 6 tests

Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.7 Plus here, though they rank higher overall. Trails Granite 4.2 30B, INTELLECT-3, EXAONE 4.5 33B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH-Vision lab +1	86.2% ✎	▁▁·▁··▄▁▃▇▇█ beats 22/24	medium · avg 54%	▇ 2.3%	small run · several runs
HMMT February 2025 lab	91.4% ✎	▁▁▁▃▁▅▅▃██▁▄ beats 20/31	medium · avg 61%	▇ 1.8%	small run
MathVista (testmini) lab	87.4% ✎	▁··▁▄▄▇█▅▆█▄ beats 45/47	medium · avg 64%	▇ 1.1%	small run
HMMT November 2025 lab	90.3% ✎	▁▁····▁▅▃█▇▁ beats 14/23	easy · avg 67%	▇ 1.1%	small run
PolyMATH lab	68.9% ✎	▁···▃▁·▇█▄▄▄ beats 21/23	hard · avg 34%	▇ 1.0%	at its level · small run
DynaMath lab	85.9% ✎	▂·····▂··█▄▆ beats 8/10	medium · avg 65%	▇ 0.9%	small run

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #53 of 147 · 15% of its score · 19 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5, Claude Opus 4.5 here, though they rank higher overall. Trails Qwen3.6 Flash, Qwen3.5 35B-A3B, Gemma 4 31B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MATH-Vision lab +1	86.2% ✎	▁▁·▁··▄▁▃▇▇█ beats 22/24	medium · avg 54%	▇ 2.3%	small run · several runs
Arena vision (style controlled)	1226.4 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 87/145	avg 1068	▇ 1.9%	blunt
CharXiv (reasoning questions) lab	77.2% ✎	▂▅▃▅▅▆▆▅▆█▆▂ beats 66/100	medium · avg 43%	▇ 1.2%	small run
MathVista (testmini) lab	87.4% ✎	▁··▁▄▄▇█▅▆█▄ beats 45/47	medium · avg 64%	▇ 1.1%	small run
MMMU (val) lab	83.9% ✎	▁····▁▁▄█▇▆▆ beats 51/52	medium · avg 58%	▇ 1.1%	small run
MMMU Pro lab	76.9% ✎	▂▃▁▃▆▇▃▇▇██▅ beats 44/52	medium · avg 47%	▇ 1.0%	at its level · small run
OmniDocBench 1.5 overall (higher is better) lab	89.8% ✎	▂··▂▂▂··▂▂█▅ beats 9/13	easy · avg 70%	▇ 0.9%	small run
MMStar lab	82.9% ✎	▃▅▂▃▂▂▆█▆▆█▅ beats 33/34	medium · avg 63%	▇ 0.8%	small run

11 more tests

test	result	vs the others in its run	difficulty	counts	why
PMC-VQA lab	63.3% ✎	▃···▃··▃▃·▃█ beats 6/7	medium · avg 39%	▇ 0.7%	at its level · small run
CountBench lab	97.0% ✎	▂▂▃▂·▂▂▃▃·▅█ beats 13/18	very easy · avg 92%	▇ 0.7%	maxed out · small run
SimpleVQA lab	61.7% ✎	▃·▃·▃·▃·█▅▃▃ beats 9/10	hard · avg 34%	▇ 0.7%	at its level · small run
MMBench-EN dev v1.1 lab	92.8% ✎	█▄···▂··▄▂▄▆ beats 13/14	easy · avg 82%	▇ 0.5%	blunt · maxed out · small run
ScreenSpot-Pro lab	70.4% ✎	▂▃▅▆██▅▂···▂ beats 19/24	medium · avg 48%	▇ 0.5%	blunt · at its level · small run
HallusionBench lab	67.6% ✎	▃▁·▃▃▃▄▅▄█▅▅ beats 27/32	medium · avg 54%	▇ 0.4%	blunt · at its level · small run
RealWorldQA lab	85.1% ✎	▂▃▂█▃██▆█▂▆▆ beats 35/38	easy · avg 72%	▇ 0.3%	blunt · small run
CC-OCR lab	81.8% ✎	▁·▂··▂▂▃▃▂▆█ beats 23/26	easy · avg 72%	▇ 0.3%	blunt · small run
OCRBench lab	92.1% ✎	▁·▃▁▆▅▂█▅█▁▃ beats 36/37	easy · avg 83%	▇ 0.2%	blunt · small run
AI2D test lab	93.3% ✎	▁▁▅▇▃▇·▄·█·▃ beats 25/28	easy · avg 85%	▇ 0.1%	blunt · small run
PureDocBench document parsing (Avg3 over Clean/Digital/Real)	74.1% ✎	▂·▁·▂▂▅█·▃▂▂ beats 18/22	medium · avg 62%	▇ 0.1%	blunt · at its level

web search▂▁▂▆▄█▃▃▁▁·▁ #58 of 80 · 5.5% of its score · 5 tests

Beats DeepSeek V4 Flash (Preview, Apr 2026), GPT-5.4 mini, Qwen3.5 397B-A17B here, though they rank higher overall. Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), Llama 4 Scout (109B-A17B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BrowseComp-ZH lab	69.9% ✎	▁▁▁▁▁▃·▁▄▅▄█ beats 21/23	hard · avg 23%	▇ 1.6%	at its level · small run
BrowseComp lab	63.8% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 29/42	hard · avg 23%	▇ 1.3%	at its level · small run
Humanity's Last Exam with tools lab	47.5% ✎	▂▂▂▁▄▂▃▆█▇▂▆ beats 24/54	hard · avg 18%	▇ 1.0%	at its level · small run
WideSearch lab	60.5% ✎	█·▂▆··▂···▂▂ beats 3/10	hard · avg 33%	▇ 0.8%	at its level · small run
SealQA Seal-0 lab	44.1% ✎	█····█·····█ beats 1/2	hard · avg 19%	▇ 0.8%	at its level · small run

other tests44 not in a task · 51% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond lab +1	86.6% ✎	▁▁▁▁▂▄▅▅███▄ beats 240/310	hard · avg 44%	▇▇▇ 6.8%	sharp · several runs
Humanity's Last Exam lab +1	25.3% ✎	▁▁▁▃▆▆▇██▆▁▃ beats 60/104	very hard · avg 5.2%	▇ 3.7%	several runs
MMLU-Pro lab +1	86.7% ✎	▁▁·▁▁▁▃▂▄▅█▃ beats 183/188	medium · avg 57%	▇ 3.1%	several runs
MedXPertQA MM lab	67.3% ✎	▂···▂·▃··▆▂█ beats 12/13	very hard · avg 31%	▇ 2.2%	sharp · at its level · small run
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	38.9% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 31/39	very hard · avg 22%	▇ 2.0%	blunt · at its level
LongBench v2 lab	60.2% ✎	▂·▁·▂▂▁▃▄▁█▄ beats 20/28	very hard · avg 35%	▇ 1.7%	at its level · small run
Polish medical board exams, bias-mitigated MCQ protocol (7-exam average)	75.1% ✎	▄·▂▂▂··█▂▄·▄ beats 13/13	hard · avg 48%	▇ 1.6%	at its level
Mystery Game Puzzles (Epoch)	17.0% ✎	█··▂▃▇▄█▂▃▁▁ beats 27/74	very hard · avg 9.4%	▇ 1.5%	sharp · too hard for it · noisy

36 more tests

test	result	vs the others in its run	difficulty	counts	why
C-Eval lab	91.9% ✎	▁···▁▁▂▁▂▂▃█ beats 30/33	medium · avg 70%	▇ 1.5%	small run
INCLUDE lab	82.8% ✎	▂····▂▂▄▄▄█▄ beats 34/36	medium · avg 55%	▇ 1.4%	small run
MMLU-Redux lab	94.0% ✎	▁▁▃▁▂▃▄▅▇▅█▄ beats 61/63	easy · avg 78%	▇ 1.2%	maxed out · small run
MMMLU lab	86.7% ✎	▂▁▁▂▃▂▄▄▅██▃ beats 40/46	medium · avg 63%	▇ 1.2%	small run
VitaBench lab	33.6% ✎	▄··▂▄▆▆█▄▄▄▂ beats 16/21	very hard · avg 11%	▇ 1.2%	at its level · small run
OJBench lab	39.5% ✎	▁···▁▃·▅▄▅██ beats 23/26	very hard · avg 13%	▇ 1.2%	at its level · small run
Global PIQA lab	88.4% ✎	▃▃··▃··▃▃▅█▅ beats 10/11	medium · avg 72%	▇ 1.0%	small run
CPTUB average (SpeakLeash, 0-5 judge scale)	87.3% ✎	▁▂▂▂▃█▃▆▅▄▂▃ beats 93/95	easy · avg 66%	▇ 1.0%	typical
IFBench (aggregation unspecified) lab	76.1% ✎	▃▇▂▄▅▅▂▄▄▆█▇ beats 66/80	medium · avg 43%	▇ 0.9%	at its level · small run
MAXIFE lab	87.9% ✎	▂·▂·▂···▆·▄█ beats 9/11	easy · avg 65%	▇ 0.9%	small run
MultiChallenge lab	61.5% ✎	▁·▁·▁▃▆▅▃▁█▅ beats 22/29	hard · avg 30%	▇ 0.9%	at its level · small run
VideoMMMU lab	82.0% ✎	▂···▂···▂▂█▆ beats 6/10	medium · avg 59%	▇ 0.9%	small run
SuperGPQA lab	67.1% ✎	▂▁▂▂▅▆▃▇▇▇█▃ beats 44/46	medium · avg 36%	▇ 0.9%	at its level · small run
MMLU-ProX lab	82.2% ✎	▁·▁▁▁▂▁▃▂█▅▄ beats 28/35	medium · avg 57%	▇ 0.9%	small run
VLMs Are Blind lab	96.7% ✎	▂·▂·····▃··█ beats 5/8	very easy · avg 89%	▇ 0.8%	maxed out · small run
LingoQA lab	80.8% ✎	▄······▄·█·█ beats 3/5	medium · avg 56%	▇ 0.8%	small run
SLAKE lab	81.6% ✎	▄·····▄▄·██▄ beats 7/7	medium · avg 59%	▇ 0.8%	small run
DeepPlanning lab	24.1% ✎	▂·····▂··▄█▄ beats 6/9	very hard · avg 8.7%	▇ 0.8%	small run
LVBench lab	74.4% ✎	▇·█▅▅▄▄··▃▁▁ beats 23/28	medium · avg 52%	▇ 0.8%	at its level · small run
Video-MME (with subtitles) lab	87.3% ✎	▁▁▁▁▁·▁▃▁▁▁█ beats 14/16	medium · avg 71%	▇ 0.8%	small run
MMVU lab	74.7% ✎	▃···▃···▃▃▅█ beats 7/8	medium · avg 48%	▇ 0.8%	small run
RefSpatial-Bench lab	69.3% ✎	▃·▃····▃▃▅▅█ beats 7/10	medium · avg 40%	▇ 0.7%	at its level · small run
FullStackBench (Chinese) lab	58.7% ✎	█······█···█ beats 2/2	hard · avg 29%	▇ 0.7%	at its level · small run
FullStackBench (English) lab	62.6% ✎	█····█·····█ beats 2/2	hard · avg 31%	▇ 0.7%	at its level · small run
ZEROBench subquestions lab	36.2% ✎	▄·▄···▄·▄██▄ beats 7/8	hard · avg 16%	▇ 0.7%	at its level · small run
MLVU lab	87.3% ✎	▂···▂···▂▄▄█ beats 10/10	easy · avg 69%	▇ 0.7%	small run
MMLongBench-Doc lab	59.0% ✎	▃····▃··▃▃▅█ beats 5/8	medium · avg 35%	▇ 0.7%	at its level · small run
ERQA lab	62.0% ✎	▃··▃··█·▃█▅▅ beats 7/12	medium · avg 36%	▇ 0.7%	at its level · small run
EmbSpatial-Bench lab	83.9% ✎	▁·····▁·▁·▃█ beats 5/10	easy · avg 66%	▇ 0.7%	small run
NOVA-63 lab	58.6% ✎	▃▃·▃···▃█▃▃█ beats 10/11	medium · avg 35%	▇ 0.6%	blunt · at its level · small run
AndroidWorld lab	66.4% ✎	▁·▁▁▂▃█▅▃▂·▂ beats 19/24	medium · avg 41%	▇ 0.6%	blunt · at its level · small run
Video-MME (without subtitles) lab	83.9% ✎	▁▃▁▃▁·█▄▄▅·▇ beats 27/27	medium · avg 67%	▇ 0.5%	blunt · small run
MVBench lab	76.6% ✎	▁··▃▄▁▃▄▄▄█▄ beats 25/26	medium · avg 66%	▇ 0.3%	blunt · at its level · small run
LIBRA Mini (Russian long context, May 2026 version)	42.7% ✎	▄···▄█·██··▄ beats 5/8	medium · avg 38%	▇ 0.2%	blunt · at its level
WMT24++ lab	78.3% ✎	▁·▁▁·▁·▁▂▆█▃ beats 16/27	easy · avg 69%	▇ 0.1%	blunt · small run
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	88.8% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 31/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · Alibaba Cloud Model Studio · $1.10/M · $0.4 in / $3.2 out per 1M tokens · 262k context · www.alibabacloud.com ↗ · price sourceUSD per 1M tokens, Singapore (International), qwen3.5-122b-a10b, 0<Token<=256K input tier; the same output price applies in thinking and non-thinking mode. China (Beijing) and Global (US Virginia/Frankfurt) deployments are cheaper ($0.115/$0.917 up to 128K) and are not used, following the International convention of other Alibaba rows. Free quota excluded. Observed 2026-09-25.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3.5 122B-A10B · needs ~74.2 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Qwen3.5-122B-A10B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Qwen3.5-122B-A10B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3.5 122B-A10B · needs ~74.2 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Qwen3.5-122B-A10B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3.5 122B-A10B · needs ~75.5 GB · from ollama.com
chat in the terminal
ollama run qwen3.5:122b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3.5 122B-A10B · needs ~69.6 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Qwen3.5-122B-A10B-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Qwen3.5-122B-A10B-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3.5-122B-A10B model… 70%, Qwen/Qwen3.5-27B model card 14%, LMArena leaderboard dataset 10% and 8 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Qwen/Qwen3.5-122B-A10B model card · lab-reported	71	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 70%	apache-2.0
Qwen/Qwen3.5-27B model card · lab-reported	7	▇▇▇▇ 14%	apache-2.0
LMArena leaderboard dataset	3	▇▇▇ 10%	CC BY 4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 2%	CC BY 4.0
Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance? (arXiv:2606.12250v1)	1	▇ 2%	CC BY-SA 4.0
Epoch AI — internal evaluations	1	▇ 1%	CC BY 4.0
SpeakLeash CPTUB (Complex Polish Text Understanding Benchmark)	1	▇ 1%	MIT
EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)	1	▇ 1%	CC BY 4.0
ai-forever LIBRA leaderboard (Russian long-context benchmark)	1	▇ 0%	MIT
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)	1	▇ 0%	CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 68.8

$ quantization evidence · 7 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	LiveCodeBench v6 code generation 0-shot (lighteval via litellm; mean of 3 seeds)	0.813 → 0.819	100.7%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	AIME 2025 0-shot (lighteval via litellm; mean of 8 seeds)	0.921 → 0.925	100.5%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	GSM8K-Platinum 0-shot (lm-evaluation-harness gsm8k_platinum_cot_llama, chat completions; mean of 3 seeds)	0.956 → 0.957	100.1%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	MMLU-Pro 0-shot (lm-evaluation-harness mmlu_pro_chat, chat completions; mean of 3 seeds)	0.87 → 0.87	100.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	MATH-500 0-shot (lighteval via litellm; mean of 3 seeds)	0.852 → 0.851	99.9%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	IFEval prompt-level strict 0-shot (lm-evaluation-harness ifeval, chat completions; mean of 3 seeds)	0.921 → 0.911	99.0%	source ↗
FP8 W8A8 dynamic (E4M3; weights static per-channel, activations dynamic per-token; llm-compressor FP8_DYNAMIC, compressed-tensors); lm_head, MoE router/shared-expert gates, linear-attention (Gated DeltaNet) and vision-encoder layers unquantized	GPQA Diamond 0-shot (lighteval via litellm; mean of 3 seeds)	0.874 → 0.86	98.5%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json