whatmodel.sh / Qwen3 30B-A3B Thinking 2507
Qwen3 30B-A3B Thinking 2507
↗ share
+ contribute
Qwen3 30B-A3B Thinking 2507 is an open-weights language model from Alibaba, released July 2025. Its published size is 30.5B parameters (3.3B active per token) and a 262k-token context window. Licence: apache-2.0.
$ how it compares
The LLMs ranked either side of Qwen3 30B-A3B Thinking 2507, out of 828 with a score, then more from Alibaba.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 223 | 61±1 | comparison only | |||
| 224 | 61±1 | comparison only | |||
| 225 | 61±1 | api $0.75/M +local | |||
| 226 | 61±1 | local | |||
| 227 | 61±1 | local | |||
| more from Alibaba | |||||
| 18 | 78±1 | api $3.00/M +local | |||
| 34 | 76±1 | api $3.00/M | |||
| 37 | 75±1 | local | |||
$ what it's good at
agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #174 of 300 · 21% of its score · 10 tests
Beats o1 (2024-12-17), o3-mini, DeepSeek R1 0528 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| τ²-bench Telecom lab +3 | 26.3% ✎ | ▃·█········█ beats 1/6 | 8.6% |
| τ-bench Retail lab +2 | 67.8% ✎ | ▁····▃·▂▇█▂▆ beats 18/28 | 3.4% |
| τ-bench Airline lab +2 | 48.0% ✎ | ▁▁▁·▅▄▁█▇▁▁▇ beats 18/28 | 2.4% |
| τ²-bench Retail lab +1 | 58.8% ✎ | ▁····▂▁▃▆█▃▁ beats 10/44 | 1.9% |
| τ²-bench Airline lab +1 | 58.0% ✎ | ▁▁·▂▁▃▂▁█▅▂▃ beats 24/42 | 1.3% |
| SWE-bench Verified | 31.0% ✎ | ▁▁▁▁▁▁▃▄█▆▁▁ beats 19/131 | 1.1% |
| BFCL v3 lab +1 | 72.4% ✎ | ▁▁▁▁▁▃▅█▆▅▄▁ beats 161/170 | 0.6% |
| τ²-bench aggregate lab | 41.9% ✎ | ▁▁▁·▃▂▄▃▃▅█▂ beats 3/37 | 0.5% |
2 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Multi-SWE-bench (scaffold per note) | 9.5% ✎ | █··█·······█ beats 1/2 | 0.5% |
| BFCL V4 (December 2025 archive) lab | 42.4% ✎ | ▁▁▁▂▇▆▆▆█▂▄▂ beats 55/111 | 0.5% |
coding▁▁▁▃▆██▇▆▄▃▁ #182 of 458 · 4.5% of its score · 3 tests
Beats Gemini 2.5 Flash here, though they rank higher overall. Trails QwQ-32B-Preview, OpenThinker 32B, Ministral 3 8B Reasoning 2512 here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| LiveCodeBench v6 (Feb–May 2025) +2 | 60.3% ✎ | ▁▁▃▁█▅▄▇▅▅▃█ beats 17/38 | 2.9% |
| SWE-bench Verified | 31.0% ✎ | ▁▁▁▁▁▁▃▄█▆▁▁ beats 19/131 | 1.1% |
| Multi-SWE-bench (scaffold per note) | 9.5% ✎ | █··█·······█ beats 1/2 | 0.5% |
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #146 of 393 · 5.3% of its score · 2 tests
Beats INTELLECT-3, Nemotron 3 Super 120B-A12B, MiniMax-M2 here, though they rank higher overall. Trails LFM2.5 1.2B Thinking, LFM2.5 8B-A1B, LFM2.5 2.6B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena-Hard v2 aggregate lab +1 | 56.0% ✎ | ▁▁··▄▃▁▃█▁▁▇ beats 8/22 | 3.9% |
| IFEval (aggregation unspecified) lab +2 | 88.9% ✎ | ▁▁▃▁▂▄▄▆█▃▂▁ beats 104/141 | 1.4% |
math▁▂▃▅▆▅▄█▄▁▁▁ #155 of 398 · 20% of its score · 10 tests
Beats MiMo-V2.5 (310B-A15B), MiniMax-M2.7, MiniMax-M2.1 here, though they rank higher overall. Trails Qwen3-VL 4B Thinking, Granite 4.2 3B, DeepSeek R1 0528 Qwen3 8B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| HMMT 2025 (edition unspecified) lab +1 | 71.4% ✎ | ▃█▇▅▇▅▁·▁··▁ beats 19/32 | 4.6% |
| AIME 2025 +1 | 81.3% ✎ | ▁·▁▁▁▂▅▆█▇▅▁ beats 100/133 | 3.2% |
| OTIS mock AIME 2024–2025 | 70.3% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 101/193 | 2.1% |
| MathArena ArXivMath January 2026 (23 problems) | 39.1% ✎ | ▂··▄▂▄█▆▆▄▄█ beats 2/23 | 1.9% |
| MathArena ArXivMath February 2026 (32 problems) | 22.7% ✎ | ▁▁▃▁█▇▄▃▁▃·▃ beats 2/24 | 1.8% |
| PolyMATH lab +1 | 52.6% ✎ | ▁···▃▁·▇█▄▄▄ beats 14/23 | 1.6% |
| AIME 2024 | 87.7% ✎ | ▁▁▁▂▁▃▁▅███▃ beats 65/85 | 1.6% |
| HMMT February 2025 lab | 63.1% ✎ | ▁▁▁▃▁▅▅▃██▁▄ beats 7/31 | 1.2% |
2 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MathArena ArXivMath December 2025 (17 problems) | 27.9% ✎ | ▃·▃▃█▁·▄·▁▁▁ beats 1/18 | 1.1% |
| HMMT November 2025 lab | 73.8% ✎ | ▁▁····▃▅▃█▇▁ beats 2/23 | 1.0% |
other tests35 not in a task · 51% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond +2 | 70.1% ✎ | ▁▁▁▁▂▄▆▄▅▇█▅ beats 105/208 | 9.1% |
| GPQA (subset unspecified) lab +1 | 73.4% ✎ | ▂▁▂▂▄▃▆▃▅█▄▃ beats 83/104 | 4.2% |
| MMLU-Pro +2 | 81.9% ✎ | ▁▁·▁▁▁▃▂▄▅█▁ beats 142/188 | 3.5% |
| MMLU-ProX lab +3 | 76.4% ✎ | ▁·▁▁▁▂▁▃▂█▅▄ beats 15/35 | 2.7% |
| SuperGPQA lab +2 | 56.8% ✎ | ▂▁▂▂▅▆▃▇▇▇█▃ beats 30/46 | 2.2% |
| INCLUDE lab +1 | 74.4% ✎ | ▂····▂▂▄▄▄█▄ beats 21/36 | 2.2% |
| MMMLU lab +1 | 78.4% ✎ | ▂▁▁▂▃▂▄▄▅██▃ beats 29/46 | 2.1% |
| MMLU-Redux lab +1 | 91.4% ✎ | ▁▁▃▁▂▃▄▅▇▅█▄ beats 42/63 | 2.0% |
27 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| OJBench lab +1 | 25.1% ✎ | ▁···▁▃·▅▄▅██ beats 13/26 | 1.9% |
| Medmarks-V weighted mean win rate (61 models, 71 configurations) | 55.9% ✎ | ▂▃▅▂▆▃█▅█▅▃▆ beats 25/34 | 1.5% |
| C-Eval lab | 87.4% ✎ | ▁···▁▁▂▁▂▂▃█ beats 18/33 | 1.4% |
| MMLU | 86.9% ✎ | ▁▁▁▁▁▂▃▅█▃▆▃ beats 167/183 | 1.4% |
| WritingBench lab +1 | 85.0% ✎ | ▁·▁▂▁··▅█▂▂▁ beats 11/20 | 1.3% |
| LongBench v2 lab | 44.8% ✎ | ▂·▁·▂▂▁▃▄▁█▄ beats 13/28 | 1.3% |
| LiveBench 2024-11-25 lab +1 | 76.8% ✎ | ▄··▂·▂·▂█▆▂▂ beats 10/13 | 1.2% |
| KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items | 27.0% ✎ | ▁▁▂▁▃▄█▆▄▅▂▃ beats 13/39 | 1.2% |
| BRIDGE clinical text, zero-shot average (87 tasks) | 41.8% ✎ | ▁▁▁▁▁▂▂▃▄▆█▃ beats 86/97 | 1.1% |
| Humanity's Last Exam | 8.7% ✎ | ▁▁▁▃▆▆▇██▆▁▃ beats 15/104 | 1.1% |
| Multi-IF (aggregation unspecified) lab +1 | 76.4% ✎ | ▁▁▁▁▁▁▃▄█▄▁▂ beats 22/41 | 1.1% |
| Global PIQA lab | 80.2% ✎ | ▃▃··▃··▃▃▅█▅ beats 4/11 | 0.9% |
| MAXIFE lab | 77.4% ✎ | ▂·▂·▂···▆·▄█ beats 3/11 | 0.9% |
| MultiChallenge lab | 46.5% ✎ | ▁·▁·▁▃▆▅▃▁█▅ beats 12/29 | 0.8% |
| VitaBench lab | 14.1% ✎ | ▄··▃▄▆▆█▄▄▄▂ beats 2/21 | 0.8% |
| IFBench (aggregation unspecified) lab | 51.5% ✎ | ▃▇▂▄▅▅▃▄▄▆█▇ beats 37/80 | 0.8% |
| Creative Writing v3 lab | 84.4% ✎ | ▂····▂▆··▆█▆ beats 6/14 | 0.7% |
| SimpleQA | 23.6% ✎ | ▄▄▄▆█▆███·▂█ beats 21/32 | 0.6% |
| KOR-Bench | 70.2% ✎ | ▄·······█··▄ beats 1/3 | 0.6% |
| NOVA-63 lab | 52.5% ✎ | ▃▃·▃···▃█▃▃█ beats 3/11 | 0.6% |
| RULER 128k | 94.5% ✎ | ▃··▅▄▇▅█▇·▃▃ beats 31/32 | 0.4% |
| Chess Puzzles (Epoch) | 8.0% ✎ | █▁▁▃▁▂▁▄▂▂▁▁ beats 54/135 | 0.4% |
| BeyondAIME | 56.0% ✎ | ▄·▂·▂▄··▄▄▄█ beats 6/15 | 0.2% |
| DeepPlanning lab | 4.9% ✎ | ▂·····▃··▄█▄ beats 1/9 | 0.2% |
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 95.2% ✎ | ▁▁▂▃▄▄█▆▃▂▂▁ beats 64/140 | 0.1% |
| miniF2F (pass@1) | 5.7% ✎ | ▃·█▅·····▅·▃ beats 0/8 | 0.1% |
| WMT24++ lab | 69.3% ✎ | ▁·▁▁·▁·▁▃▆█▃ beats 5/27 | 0.1% |
$ where to run it
Through an API
api · Alibaba Cloud Model Studio · $0.75/M · $0.2 in / $2.4 out per 1M tokens · 262k context · www.alibabacloud.com ↗ · price sourceUSD per 1M tokens, Alibaba Cloud Model Studio, Singapore (International deployment scope), qwen3-30b-a3b-thinking-2507 (thinking mode only; output = chain of thought + answer). Chinese-mainland/Global deployments priced differently; free quota, batch and context-cache discounts excluded. Currency: USD. Alibaba Cloud official Model Studio pricing page, observed 2026-09-25.
llama.cpp · recommended
version
- Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.macOS / Linux
curl -LsSf https://llama.app/install.sh | shWindowsirm https://llama.app/install.ps1 | iex - Run Qwen3 30B-A3B Thinking 2507 · needs ~18.6 GB · from huggingface.co/lmstudio-communitychat in the terminal
llama cli -hf lmstudio-community/Qwen3-30B-A3B-Thinking-2507-GGUF:Q4_K_M - Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf lmstudio-community/Qwen3-30B-A3B-Thinking-2507-GGUF:Q4_K_M
LM Studio
version
- Install LM Studio · Open the app once before using the lms command.
- Run Qwen3 30B-A3B Thinking 2507 · needs ~18.6 GB · from huggingface.co/lmstudio-community
- Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Qwen3-30B-A3B-Thinking-2507-GGUF@Q4_K_M -y - Then run lms ls to see its name, and lms chat followed by that name
Ollama
version
- Install Ollama
download it (macOS / Windows) ↗
Linuxcurl -fsSL https://ollama.com/install.sh | sh - Run Qwen3 30B-A3B Thinking 2507 · needs ~18.8 GB · from ollama.comchat in the terminal
ollama run qwen3:30b
MLX (Mac)
version
- Install MLX (Mac) · Apple silicon Macs only, text only.with uv
uv tool install mlx-lmorpip install mlx-lm - Run Qwen3 30B-A3B Thinking 2507 · needs ~17.2 GB, Mac only · from huggingface.co/lmstudio-communitychat in the terminal
mlx_lm.chat --model lmstudio-community/Qwen3-30B-A3B-Thinking-2507-MLX-4bit - Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Qwen3-30B-A3B-Thinking-2507-MLX-4bit
Official weights and documentation: huggingface.co/Qwen ↗
$ how the score was worked out · Qwen/Qwen3-Next-80B-A3B-Think… 23%, Qwen/Qwen3-30B-A3B-Thinking-2… 22%, ByteDance-Seed/Seed-OSS-36B-I… 18% and 11 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
$ quantization evidence · 12 measurements
How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.
| precision | test | full → quantized | kept | |
|---|---|---|---|---|
| INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor) | MATH-500 — published row 'Math 500'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32000 | 0.907 → 0.931 | 102.6% | source ↗ |
| INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor) | IFEval — published row 'IfEval (0-shot)'; lm-evaluation-harness task ifeval via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000) | 0.901 → 0.903 | 100.2% | source ↗ |
| INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor) | GPQA Diamond — published row 'GPQA diamond'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32000 | 0.715 → 0.716 | 100.1% | source ↗ |
| INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor) | MMLU-Pro — published row 'MMLU-Pro (0-shot)'; lm-evaluation-harness task mmlu_pro_chat via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000) | 0.807 → 0.791 | 98.0% | source ↗ |
| INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor) | AIME 2025 — published row 'AIME 2025'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32000 | 0.867 → 0.833 | 96.1% | source ↗ |
| INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor) | GSM8K-Platinum — published row 'GSM8k Platinum (0-shot)'; lm-evaluation-harness task gsm8k_platinum_cot_llama via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000) | 0.856 → 0.727 | 84.9% | source ↗ |
| INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor) | GPQA Diamond — published row 'GPQA diamond'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32000 | 0.715 → 0.724 | 101.3% | source ↗ |
| INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor) | IFEval — published row 'IfEval (0-shot)'; lm-evaluation-harness task ifeval via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000) | 0.901 → 0.901 | 100.0% | source ↗ |
| INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor) | MMLU-Pro — published row 'MMLU-Pro (0-shot)'; lm-evaluation-harness task mmlu_pro_chat via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000) | 0.807 → 0.807 | 99.9% | source ↗ |
| INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor) | MATH-500 — published row 'Math 500'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32000 | 0.907 → 0.906 | 99.9% | source ↗ |
| INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor) | GSM8K-Platinum — published row 'GSM8k Platinum (0-shot)'; lm-evaluation-harness task gsm8k_platinum_cot_llama via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000) | 0.856 → 0.838 | 98.0% | source ↗ |
| INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor) | AIME 2025 — published row 'AIME 2025'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 32000 | 0.867 → 0.842 | 97.1% | source ↗ |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json