whatmodel.sh / Step 3.5 Flash (196B-A11B)

Step 3.5 Flash (196B-A11B)

+ contribute

Step 3.5 Flash (196B-A11B) is an open-weights language model from StepFun, released February 2026. Its published size is 196B parameters (11B active per token) and a 256k-token context window. Licence: apache-2.0.

68.3±0.9#103 of 719 LLMsfrom 23 benchmark results across 5 sources

$ how it compares

The LLMs ranked either side of Step 3.5 Flash (196B-A11B), out of 828 with a score, then more from StepFun.

#	model	score	where	RAM	released
113	GPT-5.2-Codex OpenAI	69±2	comparison only		2026-01
114	LongCat-Flash-Thinking (560B-A27B) Meituan	69±1	comparison only		2025-09
115	Step 3.5 Flash (196B-A11B) StepFun	68±1	api $0.15/M +local	~118GB	2026-02
116	DeepSeek V3.2 DeepSeek	68±1	api $0.29/M +local	~110GB	2025-12
117	MiMo-V2-Omni Xiaomi	68±2	comparison only		2026-03
	more from StepFun
45	Step 5 Preview StepFun	75±1	comparison only
132	Step 3.7 Flash (198B-A11B) StepFun	67±1	api $0.44/M +local	~104GB	2026-05

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #80 of 300 · 15% of its score · 6 tests

Beats GPT-5, Grok 4.20 (0309 reasoning), GPT-5.1 here, though they rank higher overall. Trails MiniMax-M2.7 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
BrowseComp-ZH lab	66.9% ✎	▁▁▁▁▁▃·▁▄▅▄█ beats 19/23	hard · avg 18%	▇▇ 4.9%	at its level
τ²-bench aggregate lab	88.2% ✎	▄······▄█·▄▄ beats 5/5	medium · avg 46%	▇▇ 4.4%	typical
Terminal-Bench 2.0 lab	51.0% ✎	▂▂▅▅████▆▂▃▃ beats 20/36	very hard · avg 11%	▇ 3.4%	at its level · small run
BrowseComp lab	51.6% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 19/42	hard · avg 17%	▇ 2.5%	at its level · small run
SWE-bench Verified lab	74.4% ✎	▁▁▁▁▁▁▂▄█▆▁▁ beats 96/131	hard · avg 19%	—	not counted yet: no other model in the same run to compare it with
BrowseComp with context management lab	69.0% ✎	▅·▅▃·█▃·█▅·▃ beats 4/14		—	not counted yet: no other model in the same run to compare it with

coding▁▁▁▃▆██▇▆▄▃▁ #95 of 458 * · 3.2% of its score · 2 tests

Beats Grok 4.20 (0309 reasoning), Grok 4.3, Muse Glimmer 30B here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, o4-mini, Qwen3 235B-A22B Thinking 2507 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LiveCodeBench v6 (window unspecified) lab	86.4% ✎	▂▃▄▄▅██▅▄▇▃▆ beats 55/65	medium · avg 48%	▇ 3.2%	small run
SWE-bench Verified lab	74.4% ✎	▁▁▁▁▁▁▂▄█▆▁▁ beats 96/131	hard · avg 19%	—	not counted yet: no other model in the same run to compare it with

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #117 of 393 * · 7.1% of its score · 1 test

Beats LongCat 2.0 (1.6T-A48B) here, though they rank higher overall. Trails LFM2.5 2.6B, Nemotron-Cascade-8B, DeepSeek V3 0324 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1393.3 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 235/379	avg 1268	▇▇▇ 7.1%	blunt

math▁▂▃▅▆▅▄█▄▁▁▁ #77 of 398 · 61% of its score · 13 tests

Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.7 Plus here, though they rank higher overall. Trails Granite 4.2 30B, INTELLECT-3, EXAONE 4.5 33B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MathArena ArXivMath April 2026 (40 problems)	33.8% ✎	▁·····▅██▄·▁ beats 1/20	very hard · avg 7.2%	▇▇▇▇▇▇ 15%	sharp · at its level
MathArena ArXivMath February 2026 (32 problems)	32.8% ✎	▁▁▁▁█▇▄▃▁▃·▃ beats 9/24	very hard · avg 9.1%	▇▇▇ 8.5%	at its level
MathArena ArXivMath March 2026 (30 problems)	43.3% ✎	█▃▅·█▃▃·▅·▃▃ beats 0/14	very hard · avg 15%	▇▇▇ 8.1%	at its level
MathArena ArXivMath January 2026 (23 problems)	60.3% ✎	▂··▄▂▄█▆▆▄▄█ beats 13/23	hard · avg 22%	▇▇▇ 6.6%	at its level
AIME 2025 lab	97.3% ✎	▁·▁▁▁▂▅▆█▇▅▃ beats 133/133	easy · avg 67%	▇▇ 4.6%	sharp · maxed out · small run
MathArena ArXivMath December 2025 (17 problems)	41.9% ✎	▃·▃▃█▁·▄·▁▁▁ beats 10/18	hard · avg 19%	▇▇ 4.3%	at its level
MathArena BrokenArXiv April 2026 (61 problems)	13.9% ✎	▂▂▆█▃▃▂▂▂··▂ beats 2/18	very hard · avg 0.7%	▇ 3.3%	sharp · noisy
IMO-AnswerBench lab	85.4% ✎	▁▁·▁▁▁▄▅█▄▂▃ beats 33/42	medium · avg 35%	▇ 2.5%	small run

5 more tests

test	result	vs the others in its run	difficulty	counts	why
MathArena BrokenArXiv March 2026 (56 problems)	7.1% ✎	█·▆█▂·▂▂···▂ beats 3/14	very hard · avg 0.8%	▇ 2.0%	sharp · noisy
MathArena BrokenArXiv February 2026 (31 problems)	11.3% ✎	▃▂▂▃█▃▂·▃··▂ beats 6/16	very hard · avg 0.7%	▇ 1.9%	sharp · noisy
MathArena ArXivLean March 2026 (41 problems)	0.0% ✎	▃····▂·····█ beats 0/5	very hard · avg 1.0%	▇ 1.7%	too hard for it
HMMT November 2025 lab	94.0% ✎	▁▁····▁▅▃█▇▁ beats 19/23	medium · avg 65%	▇ 1.2%	noisy
HMMT February 2025 lab	98.4% ✎	▁▁▁▃▁▅▅▃██▁▄ beats 31/31	medium · avg 58%	▇ 0.9%	noisy · small run

web search██···█····██ #1 of 5 * · 7.4% of its score · 3 tests

Beats Kimi K2.5 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
BrowseComp-ZH lab	66.9% ✎	▁▁▁▁▁▃·▁▄▅▄█ beats 19/23	hard · avg 18%	▇▇ 4.9%	at its level
BrowseComp lab	51.6% ✎	▂·▄▁▂▃▆▄█▂▃▄ beats 19/42	hard · avg 17%	▇ 2.5%	at its level · small run
BrowseComp with context management lab	69.0% ✎	▅·▅▃·█▃·█▅·▃ beats 4/14		—	not counted yet: no other model in the same run to compare it with

other tests7 not in a task · 14% of its score

test	result	vs the others in its run	difficulty	counts	why
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	29.1% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 17/39	very hard · avg 22%	▇▇ 4.9%	blunt · at its level
WildClawBench	26.7% ✎	▃·▅▅█·▅█▅▃▃▃ beats 1/17	very hard · avg 11%	▇ 3.7%	at its level
ResearchRubrics lab	65.3% ✎	███··██·█··█ beats 6/6	hard · avg 31%	▇ 3.3%	at its level
GAIA (no-file subset) lab	84.5% ✎	▃▃····█▃···▃ beats 6/6	medium · avg 47%	▇ 1.7%	at its level
BrowseComp-ZH with context management lab	73.7% ✎	no other model yet		—	not counted yet: no other model in the same run to compare it with
xbench-DeepSearch (2025.05) lab	83.7% ✎	no other model yet		—	not counted yet: no other model in the same run to compare it with
xbench-DeepSearch (2025.10) lab	56.3% ✎	no other model yet		—	not counted yet: no other model in the same run to compare it with

$ where to run it

Through an API

api · $0.15/M · $0.1 in / $0.3 out per 1M tokens · 256k context · platform.stepfun.ai ↗ · price sourceUSD per 1M tokens, StepFun Open Platform (international) step-3.5-flash: input on cache miss and output (output includes reasoning). Cache-hit input $0.02/1M excluded. Currency: USD. StepFun official pricing page, observed 2026-09-25.

llama.cpp · recommended

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Step 3.5 Flash (196B-A11B) · needs ~118.1 GB · from huggingface.co/ggml-org
chat in the terminal
llama cli -hf ggml-org/Step-3.5-Flash-GGUF:Q4_K
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf ggml-org/Step-3.5-Flash-GGUF:Q4_K

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Step 3.5 Flash (196B-A11B) · needs ~118.1 GB · from huggingface.co/ggml-org
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/ggml-org/Step-3.5-Flash-GGUF@Q4_K -y
Then run lms ls to see its name, and lms chat followed by that name

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Step 3.5 Flash (196B-A11B) · needs ~110.8 GB, Mac only · from huggingface.co/mlx-community
chat in the terminal
mlx_lm.chat --model mlx-community/Step-3.5-Flash-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model mlx-community/Step-3.5-Flash-4bit

Weights and documentation: huggingface.co/stepfun-ai ↗

$ how the score was worked out · MathArena ArXivMath / BrokenA… 51%, stepfun-ai/Step-3.5-Flash mod… 33%, LMArena leaderboard dataset 7% and 2 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets	9	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 51%	CC BY-SA 4.0
stepfun-ai/Step-3.5-Flash model card · lab-reported	11	▇▇▇▇▇▇▇▇▇▇ 33%	Apache-2.0
LMArena leaderboard dataset	1	▇▇ 7%	CC BY 4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 5%	CC BY 4.0
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)	1	▇ 4%	CC BY 4.0
prior · pull toward the middle		▇ 0%
weighted average		= 68.3

not counted (5)

stepfun-ai/Step-3.5-Flash model card	BrowseComp with context management [correct]	69.0%	fewer than three models have this test yet, or this source ran it for this model alone
stepfun-ai/Step-3.5-Flash model card	BrowseComp-ZH with context management [correct]	73.7%	fewer than three models have this test yet, or this source ran it for this model alone
stepfun-ai/Step-3.5-Flash model card	SWE-bench Verified [correct]	74.4%	fewer than three models have this test yet, or this source ran it for this model alone
stepfun-ai/Step-3.5-Flash model card	xbench-DeepSearch (2025.05) [correct]	83.7%	fewer than three models have this test yet, or this source ran it for this model alone
stepfun-ai/Step-3.5-Flash model card	xbench-DeepSearch (2025.10) [correct]	56.3%	fewer than three models have this test yet, or this source ran it for this model alone

$ published speed measurements · 20

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	48.18 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=0, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 0-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	45.96 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=2048, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 2048-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	43.9 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=8192, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 8192-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	41.27 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=16384, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 16384-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	38.22 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=32768, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 32768-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	31.52 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=65536, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 65536-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
Apple Mac Studio M4 Max (128GB-class unified memory per document; GPU core count not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	8.88 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Metal · llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 (line 68); iogpu.wired_limit_mb raised to 122880 per the document; KV cache type default; row PP=262144, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 262144-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Mac Studio M4 Max memory/GPU core count not stated beyond the document's '128GB unified memory' platform statement. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	19.79 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=0, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 0-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	20.27 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=2048, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 2048-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	20.25 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=8192, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 8192-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	19.24 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=16384, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 16384-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	17.75 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=32768, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 32768-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	15.42 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=65536, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 65536-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
NVIDIA DGX Spark (GB10)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	8.39 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), CUDA (GGML_CUDA_GRAPHS=ON) · GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 llama-batched-bench -c 262150 -b 2048 -ub 1024 -npp <PP> -ntg 128 -npl 1 -ctk q8_0 -ctv q8_0 (line 119); row PP=262144, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 262144-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). DGX Spark run used q8_0 KV cache and CUDA unified-memory mode. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Windows (system/OEM and memory configuration not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	3.43 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Vulkan (Windows) · llama-batched-bench.exe -c 65540 -b 2048 -ub 2048 -npp <PP> -ntg 128 -npl 1 --mmap -t 32 (line 162); row PP=0, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 0-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Windows Vulkan result is far below the Mac/DGX rows (document TODO: 'Increase maximum context length and inference speed on AMD platforms'); treat as that document's platform state, not a Strix Halo ceiling. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Windows (system/OEM and memory configuration not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	3.4 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Vulkan (Windows) · llama-batched-bench.exe -c 65540 -b 2048 -ub 2048 -npp <PP> -ntg 128 -npl 1 --mmap -t 32 (line 162); row PP=2048, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 2048-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Windows Vulkan result is far below the Mac/DGX rows (document TODO: 'Increase maximum context length and inference speed on AMD platforms'); treat as that document's platform state, not a Strix Halo ceiling. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Windows (system/OEM and memory configuration not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	3.36 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Vulkan (Windows) · llama-batched-bench.exe -c 65540 -b 2048 -ub 2048 -npp <PP> -ntg 128 -npl 1 --mmap -t 32 (line 162); row PP=8192, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 8192-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Windows Vulkan result is far below the Mac/DGX rows (document TODO: 'Increase maximum context length and inference speed on AMD platforms'); treat as that document's platform state, not a Strix Halo ceiling. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Windows (system/OEM and memory configuration not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	3.3 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Vulkan (Windows) · llama-batched-bench.exe -c 65540 -b 2048 -ub 2048 -npp <PP> -ntg 128 -npl 1 --mmap -t 32 (line 162); row PP=16384, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 16384-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Windows Vulkan result is far below the Mac/DGX rows (document TODO: 'Increase maximum context length and inference speed on AMD platforms'); treat as that document's platform state, not a Strix Halo ceiling. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Windows (system/OEM and memory configuration not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	3.19 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Vulkan (Windows) · llama-batched-bench.exe -c 65540 -b 2048 -ub 2048 -npp <PP> -ntg 128 -npl 1 --mmap -t 32 (line 162); row PP=32768, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 32768-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Windows Vulkan result is far below the Mac/DGX rows (document TODO: 'Increase maximum context length and inference speed on AMD platforms'); treat as that document's platform state, not a Strix Halo ceiling. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Windows (system/OEM and memory configuration not stated)	Q4_K_S (stepfun-ai/Step-3.5-Flash-Int4 GGUF, 'step3.5_flash_Q4_K_S.gguf')	2.98 tok/s	StepFun llama.cpp fork (repository llama.cpp/ folder; build not stated), Vulkan (Windows) · llama-batched-bench.exe -c 65540 -b 2048 -ub 2048 -npp <PP> -ntg 128 -npl 1 --mmap -t 32 (line 162); row PP=65536, TG=128, B=1; value = S_TG (decode tokens/s over 128 generated tokens after a 65536-token prompt); single sequence Publisher's own llama-batched-bench table. S_TG column only (S_PP prompt processing and combined S not recorded). Windows Vulkan result is far below the Mac/DGX rows (document TODO: 'Increase maximum context length and inference speed on AMD platforms'); treat as that document's platform state, not a Strix Halo ceiling. The README now marks the repository's llama.cpp/ folder obsolete and points to upstream llama.cpp.	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json