Qwen3.5 0.8B

+ contribute

Qwen3.5 0.8B is an open-weights language model from Alibaba, released March 2026. Its published size is 0.87B parameters and a 262k-token context window. Licence: apache-2.0.

37.7±0.9#567 of 719 LLMsfrom 56 benchmark results across 8 sources

$ how it compares

The LLMs ranked either side of Qwen3.5 0.8B, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
658	DeepSeek-V2 Lite Chat (16B-A2.4B) DeepSeek	38±1	local	~11GB	2024-05
659	BioMistral-7B BioMistral	38±3	local	~6.1GB	2024-02
660	Qwen3.5 0.8B Alibaba	38±1	local	~2GB	2026-03
661	Code Tulu 2 7B Allen Institute for AI	38±2	local	~16GB	2023-11
662	SOLAR 10.7B Instruct v1.0 Upstage	37±1	local	~8.3GB	2023-12
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #291 of 300 * · 3.6% of its score · 2 tests

Beats QwQ-32B-Preview, DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B), Phi-4-mini 3.8B here, though they rank higher overall. Trails Granite 4.0 350M, Qwen2.5 0.5B Instruct, Granite 4.0 H 350M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
τ²-bench aggregate lab	11.6% ✎	▃▁▁·▂▂▄▃▃▅█▂ beats 0/37	medium · avg 44%	▇ 2.2%	small run
BFCL V4 (December 2025 archive) lab	25.3% ✎	▁▁▁▂▇▆▆▆█▂▄▂ beats 16/111	medium · avg 41%	▇ 1.4%	small run

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #350 of 393 * · 1.2% of its score · 1 test

Beats Gemma 4 E2B, Code Llama 70B Instruct, Phi-3-mini-4k-instruct (June 2024 update) here, though they rank higher overall. Trails Gemma 3 270M IT, SmolLM2 360M Instruct, Granite 4.0 H 350M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
IFEval (aggregation unspecified) lab	44.0% ✎	▁▃▃▁▂▄▄▆█▃▂▁ beats 4/141	easy · avg 76%	▇ 1.2%	at its level · small run

math▁▂▃▅▆▅▄█▄▁▁▁ #340 of 398 · 5.9% of its score · 3 tests

Beats GPT-3.5 Turbo (0125), DBRX Instruct, Command R Plus (04-2024) here, though they rank higher overall. Trails Qwen3 0.6B, Gemma 7B (base), LFM2 700M here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MathVista (testmini) lab	62.2% ✎	▁··▁▄▄▇█▅▆█▄ beats 17/47	easy · avg 73%	▇ 2.4%	at its level · small run
DynaMath lab	49.9% ✎	▃·····▂··█▄▆ beats 0/10	easy · avg 70%	▇ 1.8%	at its level · small run
PolyMATH lab	8.2% ✎	▃···▃▁·▇█▄▄▄ beats 0/23	hard · avg 29%	▇ 1.6%	small run

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #134 of 147 · 24% of its score · 18 tests

Beats LLaVA-OneVision Qwen2 72B (ov-sft), Gemma 3 4B, CogVLM2 Llama3 Chat 19B here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv (reasoning questions) lab	41.3% ✎	▂▅▃▅▅▆▆▅▆█▆▂ beats 20/100	medium · avg 55%	▇ 2.5%	at its level · small run
MathVista (testmini) lab	62.2% ✎	▁··▁▄▄▇█▅▆█▄ beats 17/47	easy · avg 73%	▇ 2.4%	at its level · small run
MMMU (val) lab	49.0% ✎	▁····▁▁▄█▇▆▆ beats 8/52	medium · avg 63%	▇ 2.2%	at its level · small run
ChartArena chart parsing (English, mAP_high average)	9.8% ✎	▃·▁·▃▃▁▇▃█·▄ beats 0/22	hard · avg 27%	▇ 2.2%	blunt
OmniDocBench 1.5 overall (higher is better) lab	61.0% ✎	▃··▂▂▂··▂▂█▅ beats 0/13	easy · avg 75%	▇ 1.9%	at its level · small run
MMMU Pro lab	31.2% ✎	▃▃▁▃▆▇▃▇▇██▅ beats 0/52	medium · avg 49%	▇ 1.9%	small run
MMStar lab	58.3% ✎	▃▅▂▃▂▂▆█▆▆█▅ beats 7/34	medium · avg 69%	▇ 1.7%	at its level · small run
CountBench lab	77.0% ✎	▂▂▃▃·▂▂▃▃·▅█ beats 4/18	very easy · avg 90%	▇ 1.6%	small run

10 more tests

test	result	vs the others in its run	difficulty	counts	why
PMC-VQA lab	40.4% ✎	▃···▃··▃▃·▃█ beats 0/7	medium · avg 48%	▇ 1.3%	at its level · small run
SimpleVQA lab	31.3% ✎	▃·▃·▃·▃·█▅▃▃ beats 0/10	medium · avg 40%	▇ 1.3%	small run
MMBench-EN dev v1.1 lab	69.9% ✎	█▄···▂··▄▂▄▆ beats 3/14	easy · avg 82%	▇ 1.2%	blunt · at its level · small run
BEAR-Bench (bilingual RU/EN enterprise and academic document images, overall accuracy)	10.6% ✎	▃▃·▅·▃·█·█▅▅ beats 0/14	hard · avg 24%	▇ 1.1%	typical
HallusionBench lab	53.1% ✎	▃▁·▃▃▃▄▅▄█▅▅ beats 6/32	medium · avg 59%	▇ 0.8%	blunt · at its level · small run
RealWorldQA lab	63.4% ✎	▂▃▂█▃██▆█▂▆▆ beats 6/38	easy · avg 72%	▇ 0.6%	blunt · at its level · small run
CC-OCR lab	63.2% ✎	▃·▂··▂▂▃▃▂▆█ beats 0/26	easy · avg 73%	▇ 0.6%	blunt · at its level · small run
OCRBench lab	74.5% ✎	▁·▃▁▆▅▂█▅█▁▃ beats 3/37	easy · avg 83%	▇ 0.5%	blunt · small run
AI2D test lab	69.9% ✎	▁▁▅▇▃▇·▄·█·▃ beats 2/28	easy · avg 84%	▇ 0.3%	blunt · at its level · small run
PureDocBench document parsing (Avg3 over Clean/Digital/Real)	56.0% ✎	▂·▃·▂▂▅█·▁▂▂ beats 2/22	medium · avg 62%	▇ 0.2%	blunt · at its level

other tests34 not in a task · 67% of its score

test	result	vs the others in its run	difficulty	counts	why
EuroEval Life in the UK (English knowledge MC), validation split, 5-shot, accuracy	56.3% ✎	▃·▂▂·▂·▃▃·█▃ beats 0/14	easy · avg 80%	▇▇▇ 8.0%	at its level
MMLU-Pro lab	42.3% ✎	▁▁·▁▁▁▃▂▄▅█▁ beats 32/188	medium · avg 65%	▇▇ 4.2%	at its level
C-Eval lab	50.5% ✎	▁···▁▁▂▃▂▂▃█ beats 7/33	easy · avg 75%	▇ 3.6%	at its level · small run
MMLU-Redux lab	59.5% ✎	▃▁▃▁▂▃▄▅▇▅█▄ beats 0/63	easy · avg 80%	▇ 3.3%	at its level · small run
INCLUDE lab	40.6% ✎	▂····▃▂▄▄▄█▄ beats 2/36	medium · avg 59%	▇ 3.1%	small run
MMMLU lab	44.3% ✎	▂▁▁▂▃▂▄▄▅██▃ beats 7/46	medium · avg 65%	▇ 2.8%	at its level · small run
EuroEval HellaSwag (English), validation split, 5-shot generative MC, accuracy	38.0% ✎	▃··▂▂·▂▆▆█·▄ beats 0/15	medium · avg 63%	▇ 2.6%	at its level
MAXIFE lab	39.2% ✎	▃·▂·▂···▆·▄█ beats 0/11	medium · avg 63%	▇ 2.4%	at its level · small run

26 more tests

test	result	vs the others in its run	difficulty	counts	why
Global PIQA lab	59.4% ✎	▃▃··▃··▃▃▅█▅ beats 0/11	medium · avg 70%	▇ 2.2%	small run
MMLU-ProX lab	34.6% ✎	▃·▁▁▁▂▁▃▂█▅▄ beats 0/35	medium · avg 57%	▇ 2.2%	at its level · small run
MultiChallenge lab	18.9% ✎	▃·▁·▁▃▆▅▃▁█▅ beats 0/29	hard · avg 34%	▇ 2.1%	small run
SuperGPQA lab	21.3% ✎	▃▁▂▂▅▆▃▇▇▇█▃ beats 0/46	medium · avg 39%	▇ 2.0%	small run
IFBench (aggregation unspecified) lab	21.0% ✎	▃▇▂▄▅▅▂▄▄▆█▇ beats 8/80	medium · avg 40%	▇ 2.0%	small run
VLMs Are Blind lab	59.4% ✎	▃·▂·····▃··█ beats 0/8	easy · avg 83%	▇ 1.9%	at its level · small run
MedXPertQA MM lab	17.1% ✎	▃···▂·▃··▆▂█ beats 0/13	very hard · avg 28%	▇ 1.8%	sharp · too hard for it · small run
VideoMMMU lab	44.3% ✎	▃···▂···▂▂█▆ beats 0/10	medium · avg 63%	▇ 1.8%	at its level · small run
SLAKE lab	62.6% ✎	▄·····▄▄·██▄ beats 0/7	easy · avg 72%	▇ 1.7%	at its level · small run
Video-MME (with subtitles) lab	63.8% ✎	▁▁▁▃▁·▁▃▁▁▁█ beats 3/16	easy · avg 75%	▇ 1.6%	at its level · small run
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	16.7% ✎	▁▃▂▁▃▄█▆▄▅▂▃ beats 1/39	very hard · avg 22%	▇ 1.6%	blunt · too hard for it
MLVU lab	65.6% ✎	▃···▂···▂▄▄█ beats 0/10	easy · avg 75%	▇ 1.6%	at its level · small run
MMVU lab	34.3% ✎	▃···▃···▃▃▅█ beats 0/8	medium · avg 51%	▇ 1.5%	at its level · small run
EmbSpatial-Bench lab	68.6% ✎	▃·····▁·▁·▃█ beats 0/10	easy · avg 75%	▇ 1.5%	at its level · small run
LVBench lab	45.1% ✎	▇·█▅▅▄▄··▃▁▁ beats 0/28	medium · avg 55%	▇ 1.5%	small run
RefSpatial-Bench lab	23.5% ✎	▃·▃····▃▃▅▅█ beats 0/10	medium · avg 39%	▇ 1.4%	small run
ERQA lab	34.5% ✎	▃··▃··█·▃█▅▅ beats 0/12	medium · avg 43%	▇ 1.4%	at its level · small run
MMLongBench-Doc lab	33.6% ✎	▃····▃··▃▃▅█ beats 0/8	medium · avg 44%	▇ 1.4%	at its level · small run
NOVA-63 lab	42.4% ✎	▃▃·▃···▃█▃▃█ beats 1/11	medium · avg 46%	▇ 1.3%	blunt · at its level · small run
ZEROBench subquestions lab	12.9% ✎	▄·▄···▄·▄██▄ beats 0/8	hard · avg 19%	▇ 1.2%	small run
Video-MME (without subtitles) lab	57.7% ✎	▁▃▃▃▁·█▄▄▅·▇ beats 3/27	medium · avg 68%	▇ 1.1%	blunt · at its level · small run
LongBench v2 lab	26.1% ✎	▃·▁·▂▂▁▃▄▁█▄ beats 0/28	very hard · avg 35%	▇ 0.9%	too hard for it · noisy · small run
MVBench lab	55.8% ✎	▃··▃▄▁▃▄▄▄█▄ beats 0/26	medium · avg 66%	▇ 0.6%	blunt · at its level · small run
LIBRA Mini (Russian long context, May 2026 version)	35.5% ✎	▄···▄█·██··▄ beats 1/8	medium · avg 38%	▇ 0.3%	blunt · at its level
WMT24++ lab	27.2% ✎	▃·▁▁·▁·▁▂▆█▃ beats 0/27	easy · avg 66%	▇ 0.3%	blunt · at its level · small run
GPQA (subset unspecified) lab	11.9% ✎	▂▁▂▂▄▃▆▃▅█▄▃ beats 0/104	very hard · avg 33%	—	failed run: 11.9% on GPQA (4 options, chance 25%) in thinking mode; probably unterminated thinking or unparsed answers rather than below-random knowledge. The next-lowest model on this benchmark scores 25.2%.

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3.5 0.8B · needs ~0.5 GB · from huggingface.co/lmstudio-community
chat in the terminal
llama cli -hf lmstudio-community/Qwen3.5-0.8B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Qwen3.5-0.8B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3.5 0.8B · needs ~0.5 GB · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Qwen3.5-0.8B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3.5 0.8B · needs ~1.54 GB · from ollama.com
chat in the terminal
ollama run qwen3.5:0.8b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3.5 0.8B · needs ~0.6 GB, Mac only · from huggingface.co/lmstudio-community
chat in the terminal
mlx_lm.chat --model lmstudio-community/Qwen3.5-0.8B-MLX-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Qwen3.5-0.8B-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3.5-2B model card 79%, EuroEval leaderboards (result… 11%, Qwen/Qwen3.5-0.8B model card 4% and 5 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Qwen/Qwen3.5-2B model card · lab-reported	48	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 79%	apache-2.0
EuroEval leaderboards (results collection), English generative leaderboard	2	▇▇▇ 11%	MIT
Qwen/Qwen3.5-0.8B model card · lab-reported	1	▇ 4%	apache-2.0
ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)	1	▇ 2%	CC BY 4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 2%	CC BY 4.0
BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)	1	▇ 1%	CC BY 4.0
ai-forever LIBRA leaderboard (Russian long-context benchmark)	1	▇ 0%	MIT
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)	1	▇ 0%	CC BY 4.0
prior · pull toward the middle		▇ 0%
weighted average		= 37.7

not counted (1)

Qwen/Qwen3.5-2B model card

GPQA (subset unspecified) [correct]

11.9%

failed run: 11.9% on GPQA (4 options, chance 25%) in thinking mode; probably unterminated thinking or unparsed answers rather than below-random knowledge. The next-lowest model on this benchmark scores 25.2%.

$ published speed measurements · 15

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	113.4 tok/s	omlx (MLX; SSD prefix cache, in-memory tier disabled; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	105.5 tok/s	llama.cpp server (Metal; --parallel 4 slots; build not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	100.9 tok/s	vllm-metal (Metal; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	95.3 tok/s	vllm-mlx (MLX; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	93.9 tok/s	mlx_lm server (MLX; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	77 tok/s	omlx (MLX; SSD prefix cache, in-memory tier disabled; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	76.2 tok/s	vllm-metal (Metal; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	67.6 tok/s	vllm-mlx (MLX; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	BF16 (paper's main-track/fidelity protocol; Appendix E caption does not restate dtype)	66.9 tok/s	llama.cpp server (Metal; --parallel 4 slots; build not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate) Concurrency-1 cell; 100/100 requests succeeded. Appendix E speed tables do not restate dtype; the paper's main track and cross-model fidelity protocol are BF16.	source ↗
NVIDIA DGX Spark (GB10)	BF16	106.5 tok/s	llama.cpp (CUDA build, GGUF BF16 weights; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench DGX Spark reference track (--platform dgxspark); identical prompts and concurrency grid as the Apple track; n=100 per level; concurrency 1; output throughput = generated tokens / sweep wall time (includes prefill); runs of 2026-07-03 to 2026-07-05 Chat split, concurrency 1.	source ↗
NVIDIA DGX Spark (GB10)	BF16	77.5 tok/s	llama.cpp (CUDA build, GGUF BF16 weights; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench DGX Spark reference track (--platform dgxspark); identical prompts and concurrency grid as the Apple track; n=100 per level; concurrency 1; output throughput = generated tokens / sweep wall time (includes prefill); runs of 2026-07-03 to 2026-07-05 Agent split, concurrency 1.	source ↗
NVIDIA DGX Spark (GB10)	BF16	94.2 tok/s	vLLM (CUDA, Safetensors BF16; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench DGX Spark reference track (--platform dgxspark); identical prompts and concurrency grid as the Apple track; n=100 per level; concurrency 1; output throughput = generated tokens / sweep wall time (includes prefill); runs of 2026-07-03 to 2026-07-05 Chat split, concurrency 1.	source ↗
NVIDIA DGX Spark (GB10)	BF16	73.4 tok/s	vLLM (CUDA, Safetensors BF16; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench DGX Spark reference track (--platform dgxspark); identical prompts and concurrency grid as the Apple track; n=100 per level; concurrency 1; output throughput = generated tokens / sweep wall time (includes prefill); runs of 2026-07-03 to 2026-07-05 Agent split, concurrency 1.	source ↗
NVIDIA DGX Spark (GB10)	BF16	95 tok/s	SGLang (CUDA, Safetensors BF16; version not stated) · chat split (OpenOrca + CNN/DailyMail, 21-3.5K input tokens); SiliconBench DGX Spark reference track (--platform dgxspark); identical prompts and concurrency grid as the Apple track; n=100 per level; concurrency 1; output throughput = generated tokens / sweep wall time (includes prefill); runs of 2026-07-03 to 2026-07-05 Chat split, concurrency 1.	source ↗
NVIDIA DGX Spark (GB10)	BF16	79.4 tok/s	SGLang (CUDA, Safetensors BF16; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench DGX Spark reference track (--platform dgxspark); identical prompts and concurrency grid as the Apple track; n=100 per level; concurrency 1; output throughput = generated tokens / sweep wall time (includes prefill); runs of 2026-07-03 to 2026-07-05 Agent split, concurrency 1.	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json