Qwen3.8 27B

+ contribute

Qwen3.8 27B is an open-weights language model from Alibaba, released August 2026. Its published size is 27.3B parameters and a 262k-token context window. Licence: apache-2.0.

73.4±0.8#55 of 719 LLMsfrom 33 benchmark results across 9 sources

$ how it compares

The LLMs ranked either side of Qwen3.8 27B, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
57	GLM-5.2 Z.ai	74±1	api $2.15/M +local	~121GB	2026-06
58	Dola Seed 2.0 Pro ByteDance	74±2	comparison only
59	Qwen3.8 27B Alibaba	73±1	api $1.60/M +local	~18GB	2026-08
60	Grok 4.20 Multi-Agent (0309) SpaceXAI (xAI)	73±2	api $1.56/M		2026-03
61	Kimi K2.6 Moonshot AI	73±1	api $1.71/M +local	~317GB	2026-04
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #45 of 300 · 29% of its score · 8 tests

Beats Gemini 3.1 Pro (Preview), Muse Spark 1.2, Muse Spark 1.1 here, though they rank higher overall. Trails Claude Sonnet 4.6 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena Agent (October 2026 aggregate IPS)	-0.0166 IPS ✎	▁▁▁▄▂▃▇█▂▂▁▃ beats 14/49	avg -0.312 IPS	▇▇▇▇ 10%	sharp
SWE-bench Pro public set, Qwen-refined lab +1	61.7% ✎	▂·····▂·▂█▃▅ beats 10/12	hard · avg 18%	▇▇ 5.2%	at its level · several runs
Terminal-Bench 2.1 lab	73.0% ✎	▁·▁·▁▁▄▃▄█▃▇ beats 37/56	hard · avg 16%	▇ 3.5%	at its level · small run
SWE-bench Multilingual lab	73.8% ✎	▁··▁▃▂▅▇█▃▃▁ beats 40/60	hard · avg 29%	▇ 3.2%	typical
Thinkingbox-bench pass@1 (507 stateful business-workflow agent tasks, N=20)	51.7% ✎	▅··▃·▃·█▃▅██ beats 9/15	very hard · avg 4.9%	▇ 2.4%	at its level
OSWorld-Verified lab	84.3% ✎	█▄·▂▆▂▄▂▆▄▆█ beats 23/25	hard · avg 27%	▇ 2.0%	at its level · small run
WebArena-Verified lab	64.8% ✎	█···█······█ beats 2/2	hard · avg 22%	▇ 1.3%	at its level · small run
DeepSWE v1.1 (resolved) lab	42.2% ✎	▂····▃▁▁▆██▆ beats 2/28	hard · avg 15%	▇ 0.8%	at its level · noisy

coding▁▁▁▃▆██▇▆▄▃▁ #31 of 458 · 27% of its score · 6 tests

Beats GPT-5.5, GPT-5.6 Terra, Claude Opus 4.8 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
Arena WebDev (Code Arena)	1591.2 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 94/119	avg 981	▇▇▇▇ 9.1%	sharp
LiveCodeBench v6 (window unspecified) lab +1	90.3% ✎	▃▅█▅▅▃▅···▃█ beats 14/16	medium · avg 46%	▇▇ 5.3%	small run · several runs
SWE-bench Pro public set, Qwen-refined lab +1	61.7% ✎	▂·····▂·▂█▃▅ beats 10/12	hard · avg 18%	▇▇ 5.2%	at its level · several runs
Luau Leetcode fully solved (Torpedo Software)	64.9% ✎	▁▁▁▁·▁▂▃▅▃█▇ beats 16/49	hard · avg 18%	▇▇ 3.8%	at its level
SWE-bench Multilingual lab	73.8% ✎	▁··▁▃▂▅▇█▃▃▁ beats 40/60	hard · avg 29%	▇ 3.2%	typical
DeepSWE v1.1 (resolved) lab	42.2% ✎	▂····▃▁▁▆██▆ beats 2/28	hard · avg 15%	▇ 0.8%	at its level · noisy

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #68 of 393 * · 5.1% of its score · 1 test

Beats GPT-5.2 here, though they rank higher overall. Trails Nemotron Cascade 2 30B-A3B, Gemma 4 31B, Gemini 2.5 Pro here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1437.7 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 296/379	avg 1268	▇▇ 5.1%	blunt

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #39 of 147 · 15% of its score · 6 tests

Beats MiMo-V2.6-Pro, MiMo-V2.6-Flash (309B-A15B), DeepSeek V4 Flash Vision (Exp) here, though they rank higher overall. Trails Qwen3.6 Flash, Qwen3.6 Plus, Qwen3.5 397B-A17B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv (reasoning questions) lab +1	83.7% ✎	▄▂▄▆▄▄▆··▄█▂ beats 18/21	medium · avg 39%	▇▇▇ 7.9%	several runs
Arena vision (style controlled)	1242.8 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 96/145	avg 1068	▇ 3.5%	blunt
SciDocBench scientific-document workflows (overall, 496 instances)	52.9% ✎	▃▃···█▃▅▃·▃▅ beats 5/11	hard · avg 26%	▇ 1.7%	blunt · at its level
OmniDocBench 1.5 overall (higher is better) lab	91.1% ✎	▂··▂▂▂··▂▂█▅ beats 12/13	easy · avg 69%	▇ 1.5%	small run
RealWorldQA lab	85.9% ✎	▂▃▂█▃██▆█▂▆▆ beats 37/38	easy · avg 72%	▇ 0.5%	blunt · small run
PureDocBench document parsing (Avg3 over Clean/Digital/Real)	77.6% ✎	▂·▁·▂▂▅█·▁▃▂ beats 20/22	medium · avg 62%	▇ 0.1%	blunt · at its level

other tests9 not in a task · 33% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond lab +1	89.2% ✎	▁▁▁▁▂▄▅▅███▄ beats 259/310	hard · avg 41%	▇▇▇▇▇ 12%	sharp · several runs
Humanity's Last Exam lab +1	30.8% ✎	▁▁▁▃▆▆▇██▆▁▃ beats 75/104	very hard · avg 5.1%	▇▇▇ 6.4%	at its level · several runs
IFBench (aggregation unspecified) lab +1	79.5% ✎	▃▇▂▄▅▅▂▄▄▆█▇ beats 73/80	medium · avg 42%	▇▇ 3.8%	several runs
BRIDGE clinical text, zero-shot average (87 tasks)	46.6% ✎	▁▁▁▁▁▂▂▃▄▆█▃ beats 95/97	hard · avg 32%	▇ 2.9%	blunt · at its level
NL2Repo lab	42.3% ✎	▂▂·▂▂▄█▆▆▂▄▆ beats 11/21	very hard · avg 9.6%	▇ 2.5%	at its level · small run
Open Persian LLM Leaderboard v2 average (21 tasks)	76.0% ✎	▂▄▂▃█▄▃▆▃▅▂▃ beats 79/81	medium · avg 58%	▇ 1.4%	blunt
RobloxQA v2.0 multiple choice (Torpedo Software)	90.8% ✎	▁▁▃▁█▄▃▃▆▂▂▂ beats 18/49	easy · avg 78%	▇ 1.3%	typical
ERQA lab	65.5% ✎	▃··▃··█·▃█▅▅ beats 10/12	hard · avg 35%	▇ 1.3%	at its level · small run

1 more test

test	result	vs the others in its run	difficulty	counts	why
AndroidWorld lab	81.9% ✎	▁·▁▁▂▃█▅▂▂·▃ beats 24/24	medium · avg 46%	▇ 1.0%	blunt · small run

$ where to run it

Through an API

api · Groq · $1.60/M · $0.8 in / $4 out per 1M tokens · 262k context · console.groq.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3.8 27B · needs ~19 GB · from huggingface.co/ggml-org
chat in the terminal
llama cli -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3.8 27B · needs ~19 GB · from huggingface.co/ggml-org
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3.8 27B · needs ~16.88 GB · from ollama.com
chat in the terminal
ollama run qwen3.8:27b

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3.8 27B · needs ~16.1 GB, Mac only · from huggingface.co/lmstudio-community
chat in the terminal
mlx_lm.chat --model lmstudio-community/Qwen3.8-27B-MLX-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Qwen3.8-27B-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3.8-27B model card 31%, LMArena leaderboard dataset 28%, Qwen (Alibaba) vendor-reporte… 28% and 6 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Qwen/Qwen3.8-27B model card · lab-reported	14	▇▇▇▇▇▇▇▇▇ 31%	apache-2.0
LMArena leaderboard dataset	4	▇▇▇▇▇▇▇▇ 28%	CC BY 4.0
Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · lab-reported	8	▇▇▇▇▇▇▇▇ 28%	Factual vendor-reported score (not licensed content)
Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode	2	▇▇ 5%	MIT
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)	1	▇ 3%	MIT
One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflows (arXiv:2608.19741v4)	1	▇ 2%	CC BY 4.0
SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding (arXiv:2609.05141v2)	1	▇ 2%	CC BY 4.0
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)	1	▇ 1%	Apache-2.0
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)	1	▇ 0%	CC BY 4.0
prior · pull toward the middle		▇ 0%
weighted average		= 73.4

$ published speed measurements · 4

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB	Q4_K_M	20.42 tok/s	Ollama 0.32.13 system service, Ollama Vulkan RADV · Ollama API generation rate (eval tokens/s) as tabulated; prompt 45 tokens; context window setting 4096; one request at a time unless notes say otherwise; generated-token count not tabulated Ollama API eval rate from the guide's structured table; warm-run means where the note says so. Row date 2026-08-15, status measured-local, kernel 7.0.0-28-generic, driver stack Mesa 26.1.6 kisak. Author note: Official 27.3B dense multimodal Ollama route; 9 warm API repeats after one cold load; 19.85-20.79 t/s generation range; 100% GPU reported; image text tool-call thinking and exact-retrieval smokes passed through 50059 evaluated prompt tokens; a 56051-token attempt caused a recoverable Vulkan DeviceLost; broad quality and advertised 262K context remain unqualified.	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	GGUF Q8_0	3.8 tok/s	llama.cpp server (Metal; --parallel 4 slots; build not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate); runs of 2026-08-24/25 Concurrency-1 cell of the 64GB memory-ceiling study; 100/100 requests succeeded. omlx streams ~2 tokens per SSE chunk (paper caveat on its ITL, not throughput).	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	MLX 8-bit (mlx-community conversion)	3.9 tok/s	vllm-metal (Metal; version not stated) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate); runs of 2026-08-24/25 Concurrency-1 cell of the 64GB memory-ceiling study; 100/100 requests succeeded. omlx streams ~2 tokens per SSE chunk (paper caveat on its ITL, not throughput).	source ↗
Apple M5 Pro (64GB; CPU/GPU core counts not stated)	MLX 8-bit (mlx-community conversion)	4.7 tok/s	omlx (MLX; SSD prefix cache, in-memory tier disabled; version not stated); omlx default config (SSD-paged prefix cache, fresh cache dir + restart per level, no memory sidecar) · agent split (BFCL V3, Hermes, ClawsBench; 0.3K-8.9K input tokens; agent context raised to 16384 for llama.cpp/ollama/vllm-metal); SiliconBench harness (benchmark commit 616aa51c); 100 prompts per level after 3 warmup requests; closed loop at concurrency 1 (one request in flight); temperature 0, non-thinking generation; output throughput = total generated tokens / measured sweep wall time, so it includes prefill/TTFT and HTTP overhead (not a pure decode rate); runs of 2026-08-24/25 Concurrency-1 cell of the 64GB memory-ceiling study; 100/100 requests succeeded. omlx streams ~2 tokens per SSE chunk (paper caveat on its ITL, not throughput).	source ↗

$ quantization evidence · 16 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precision	test	full → quantized	kept
INT4 W4A16 (AWQ smoothing + GPTQ, symmetric, group 128; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP, linear-attention in_proj_a/b and some linear_attn modules unquantized per config; llm-compressor)	GSM8K-Platinum strict-match (lm-evaluation-harness, vLLM; mean of 3 seeds)	0.958 → 0.968	101.1%	source ↗
INT4 W4A16 (AWQ smoothing + GPTQ, symmetric, group 128; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP, linear-attention in_proj_a/b and some linear_attn modules unquantized per config; llm-compressor)	IFEval prompt-level strict 0-shot (lm-evaluation-harness, vLLM; mean of 3 seeds)	0.922 → 0.919	99.7%	source ↗
INT4 W4A16 (AWQ smoothing + GPTQ, symmetric, group 128; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP, linear-attention in_proj_a/b and some linear_attn modules unquantized per config; llm-compressor)	MATH-500 pass@1 (lighteval, vLLM; mean of 3 seeds)	0.837 → 0.833	99.5%	source ↗
INT4 W4A16 (AWQ smoothing + GPTQ, symmetric, group 128; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP, linear-attention in_proj_a/b and some linear_attn modules unquantized per config; llm-compressor)	MMLU-Pro exact-match (lm-evaluation-harness, vLLM; mean of 3 seeds)	0.845 → 0.835	98.8%	source ↗
INT4 W4A16 (AWQ smoothing + GPTQ, symmetric, group 128; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP, linear-attention in_proj_a/b and some linear_attn modules unquantized per config; llm-compressor)	AIME 2025 pass@1 (lighteval, vLLM; mean of 8 seeds)	0.954 → 0.942	98.7%	source ↗
INT4 W4A16 (AWQ smoothing + GPTQ, symmetric, group 128; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP, linear-attention in_proj_a/b and some linear_attn modules unquantized per config; llm-compressor)	GPQA Diamond pass@1 (lighteval, vLLM; mean of 3 seeds)	0.892 → 0.879	98.5%	source ↗
Mixed NVFP4/FP8 (MLP projections NVFP4 group 16; attention/linear-attention projections and last 8 MLP layers FP8 dynamic; AWQ smoothing + GPTQ; FP8 KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	IFEval 0-shot (lm-evaluation-harness, vLLM; reported metric variant not stated; mean of 3 seeds)	0.912 → 0.92	100.9%	source ↗
Mixed NVFP4/FP8 (MLP projections NVFP4 group 16; attention/linear-attention projections and last 8 MLP layers FP8 dynamic; AWQ smoothing + GPTQ; FP8 KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	MATH-500 0-shot (lighteval, vLLM; mean of 3 seeds)	0.837 → 0.843	100.7%	source ↗
Mixed NVFP4/FP8 (MLP projections NVFP4 group 16; attention/linear-attention projections and last 8 MLP layers FP8 dynamic; AWQ smoothing + GPTQ; FP8 KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	GSM8K-Platinum 0-shot (lm-evaluation-harness, vLLM; mean of 3 seeds)	0.963 → 0.967	100.5%	source ↗
Mixed NVFP4/FP8 (MLP projections NVFP4 group 16; attention/linear-attention projections and last 8 MLP layers FP8 dynamic; AWQ smoothing + GPTQ; FP8 KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	GPQA Diamond 0-shot (lighteval, vLLM; mean of 3 seeds)	0.896 → 0.892	99.6%	source ↗
Mixed NVFP4/FP8 (MLP projections NVFP4 group 16; attention/linear-attention projections and last 8 MLP layers FP8 dynamic; AWQ smoothing + GPTQ; FP8 KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	AIME 2025 0-shot (lighteval, vLLM; mean of 8 seeds)	0.967 → 0.95	98.3%	source ↗
MXFP4 W4A4 (FP4 weights and activations, group 32; AWQ smoothing + GPTQ; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	MATH-500 pass@1 (lighteval, vLLM; mean of 3 seeds)	0.837 → 0.847	101.2%	source ↗
MXFP4 W4A4 (FP4 weights and activations, group 32; AWQ smoothing + GPTQ; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	IFEval prompt-level strict 0-shot (lm-evaluation-harness, vLLM; mean of 3 seeds)	0.912 → 0.909	99.7%	source ↗
MXFP4 W4A4 (FP4 weights and activations, group 32; AWQ smoothing + GPTQ; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	GSM8K-Platinum strict-match 0-shot (lm-evaluation-harness, vLLM; mean of 3 seeds)	0.963 → 0.952	98.9%	source ↗
MXFP4 W4A4 (FP4 weights and activations, group 32; AWQ smoothing + GPTQ; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	GPQA Diamond pass@1 (lighteval, vLLM; mean of 3 seeds)	0.896 → 0.877	97.9%	source ↗
MXFP4 W4A4 (FP4 weights and activations, group 32; AWQ smoothing + GPTQ; FP8 static per-tensor KV cache; lm_head, vision encoder, MTP and linear-attention in_proj_a/b unquantized; llm-compressor)	AIME 2025 pass@1 (lighteval, vLLM; mean of 8 seeds)	0.967 → 0.925	95.7%	source ↗

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json