whatmodel.sh / Qwen3 30B-A3B Thinking 2507

Qwen3 30B-A3B Thinking 2507 is an open-weights language model from Alibaba, released July 2025. Its published size is 30.5B parameters (3.3B active per token) and a 262k-token context window. Licence: apache-2.0.

60.6±0.5#187 of 719 LLMsfrom 93 benchmark results across 14 sources

$ how it compares

The LLMs ranked either side of Qwen3 30B-A3B Thinking 2507, out of 828 with a score, then more from Alibaba.

#modelscorewhereRAMreleased
223o1-mini OpenAI61±1comparison only2024-09
224Gemini 2.0 Pro Experimental (02-05) Google61±1comparison only2025-02
225Qwen3 30B-A3B Thinking 2507 Alibaba61±1api $0.75/M +local~20GB2025-07
226Qwen3.5 4B Alibaba61±1local~4.3GB2026-03
227Nemotron-Cascade-14B-Thinking NVIDIA61±1local~10GB2025-12
more from Alibaba
18Qwen3.8 Max (2.4T-A95B) Alibaba78±1api $3.00/M +local~380GB2026-08
34Qwen3.8 Max (0902) Alibaba76±1api $3.00/M2026-09
37Qwen3.8-Flash-Next Alibaba75±1local~59GB2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #174 of 300 · 21% of its score · 10 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 0528 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
τ²-bench Telecom lab +326.3% ✎▃·█········█ beats 1/6very hard · avg 10%▇▇▇ 8.6%sharp · at its level · several runs
τ-bench Retail lab +267.8% ✎▁····▃·▂▇█▂▆ beats 18/28medium · avg 36%▇ 3.4%at its level · several runs
τ-bench Airline lab +248.0% ✎▁▁▁·▅▄▁█▇▁▁▇ beats 18/28hard · avg 25%▇ 2.4%at its level · small run · several runs
τ²-bench Retail lab +158.8% ✎▁····▂▁▃▆█▃▁ beats 10/44medium · avg 37%▇ 1.9%at its level · several runs
τ²-bench Airline lab +158.0% ✎▁▁·▂▁▃▂▁█▅▂▃ beats 24/42hard · avg 32%▇ 1.3%at its level · several runs
SWE-bench Verified31.0% ✎▁▁▁▁▁▁▃▄█▆▁▁ beats 19/131hard · avg 17%▇ 1.1%sharp · at its level · small run
BFCL v3 lab +172.4% ✎▁▁▁▁▁▃▅█▆▅▄▁ beats 161/170medium · avg 59%▇ 0.6%blunt · at its level · several runs
τ²-bench aggregate lab41.9% ✎▁▁▁·▃▂▄▃▃▅█▂ beats 3/37medium · avg 37%▇ 0.5%at its level · noisy
2 more tests
testresultvs the others in its rundifficultycountswhy
Multi-SWE-bench (scaffold per note)9.5% ✎█··█·······█ beats 1/2very hard · avg 5.7%▇ 0.5%small run
BFCL V4 (December 2025 archive) lab42.4% ✎▁▁▁▂▇▆▆▆█▂▄▂ beats 55/111medium · avg 36%▇ 0.5%at its level
coding▁▁▁▃▆██▇▆▄▃▁ #182 of 458 · 4.5% of its score · 3 tests

Beats Gemini 2.5 Flash here, though they rank higher overall. Trails QwQ-32B-Preview, OpenThinker 32B, Ministral 3 8B Reasoning 2512 here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LiveCodeBench v6 (Feb–May 2025) +260.3% ✎▁▁▃▁█▅▄▇▅▅▃█ beats 17/38medium · avg 40%▇ 2.9%at its level · several runs
SWE-bench Verified31.0% ✎▁▁▁▁▁▁▃▄█▆▁▁ beats 19/131hard · avg 17%▇ 1.1%sharp · at its level · small run
Multi-SWE-bench (scaffold per note)9.5% ✎█··█·······█ beats 1/2very hard · avg 5.7%▇ 0.5%small run
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #146 of 393 · 5.3% of its score · 2 tests

Beats INTELLECT-3, Nemotron 3 Super 120B-A12B, MiniMax-M2 here, though they rank higher overall. Trails LFM2.5 1.2B Thinking, LFM2.5 8B-A1B, LFM2.5 2.6B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Arena-Hard v2 aggregate lab +156.0% ✎▁▁··▄▃▁▃█▁▁▇ beats 8/22hard · avg 28%▇▇ 3.9%sharp · at its level · several runs
IFEval (aggregation unspecified) lab +288.9% ✎▁▁▃▁▂▄▄▆█▃▂▁ beats 104/141easy · avg 78%▇ 1.4%several runs
math▁▂▃▅▆▅▄█▄▁▁▁ #155 of 398 · 20% of its score · 10 tests

Beats MiMo-V2.5 (310B-A15B), MiniMax-M2.7, MiniMax-M2.1 here, though they rank higher overall. Trails Qwen3-VL 4B Thinking, Granite 4.2 3B, DeepSeek R1 0528 Qwen3 8B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HMMT 2025 (edition unspecified) lab +171.4% ✎▃█▇▅▇▅▁·▁··▁ beats 19/32hard · avg 33%▇▇ 4.6%sharp · at its level · several runs
AIME 2025 +181.3% ✎▁·▁▁▁▂▅▆█▇▅▁ beats 100/133medium · avg 55%▇ 3.2%sharp · small run · several runs
OTIS mock AIME 2024–202570.3% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 101/193very hard · avg 8.5%▇ 2.1%sharp · at its level · noisy
MathArena ArXivMath January 2026 (23 problems)39.1% ✎▂··▄▂▄█▆▆▄▄█ beats 2/23hard · avg 22%▇ 1.9%at its level
MathArena ArXivMath February 2026 (32 problems)22.7% ✎▁▁▃▁█▇▄▃▁▃·▃ beats 2/24very hard · avg 9.1%▇ 1.8%typical
PolyMATH lab +152.6% ✎▁···▃▁·▇█▄▄▄ beats 14/23hard · avg 32%▇ 1.6%at its level · several runs
AIME 202487.7% ✎▁▁▁▂▁▃▁▅███▃ beats 65/85easy · avg 68%▇ 1.6%sharp
HMMT February 2025 lab63.1% ✎▁▁▁▃▁▅▅▃██▁▄ beats 7/31medium · avg 48%▇ 1.2%at its level
2 more tests
testresultvs the others in its rundifficultycountswhy
MathArena ArXivMath December 2025 (17 problems)27.9% ✎▃·▃▃█▁·▄·▁▁▁ beats 1/18hard · avg 19%▇ 1.1%at its level
HMMT November 2025 lab73.8% ✎▁▁····▃▅▃█▇▁ beats 2/23medium · avg 61%▇ 1.0%typical
other tests35 not in a task · 51% of its score
testresultvs the others in its rundifficultycountswhy
GPQA Diamond +270.1% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 105/208hard · avg 40%▇▇▇▇ 9.1%sharp · at its level · several runs
GPQA (subset unspecified) lab +173.4% ✎▂▁▂▂▄▃▆▃▅█▄▃ beats 83/104hard · avg 49%▇▇ 4.2%sharp · at its level · several runs
MMLU-Pro +281.9% ✎▁▁·▁▁▁▃▂▄▅█▁ beats 142/188medium · avg 61%▇ 3.5%several runs
MMLU-ProX lab +376.4% ✎▁·▁▁▁▂▁▃▂█▅▄ beats 15/35medium · avg 60%▇ 2.7%several runs
SuperGPQA lab +256.8% ✎▂▁▂▂▅▆▃▇▇▇█▃ beats 30/46medium · avg 39%▇ 2.2%at its level · several runs
INCLUDE lab +174.4% ✎▂····▂▂▄▄▄█▄ beats 21/36medium · avg 58%▇ 2.2%at its level · several runs
MMMLU lab +178.4% ✎▂▁▁▂▃▂▄▄▅██▃ beats 29/46medium · avg 63%▇ 2.1%at its level · several runs
MMLU-Redux lab +191.4% ✎▁▁▃▁▂▃▄▅▇▅█▄ beats 42/63easy · avg 82%▇ 2.0%several runs
27 more tests
testresultvs the others in its rundifficultycountswhy
OJBench lab +125.1% ✎▁···▁▃·▅▄▅██ beats 13/26very hard · avg 13%▇ 1.9%at its level · several runs
Medmarks-V weighted mean win rate (61 models, 71 configurations)55.9% ✎▂▃▅▂▆▃█▅█▅▃▆ beats 25/34medium · avg 42%▇ 1.5%blunt · at its level
C-Eval lab87.4% ✎▁···▁▁▂▁▂▂▃█ beats 18/33medium · avg 73%▇ 1.4%typical
MMLU86.9% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 167/183medium · avg 72%▇ 1.4%typical
WritingBench lab +185.0% ✎▁·▁▂▁··▅█▂▂▁ beats 11/20easy · avg 70%▇ 1.3%several runs
LongBench v2 lab44.8% ✎▂·▁·▂▂▁▃▄▁█▄ beats 13/28very hard · avg 35%▇ 1.3%at its level
LiveBench 2024-11-25 lab +176.8% ✎▄··▂·▂·▂█▆▂▂ beats 10/13medium · avg 60%▇ 1.2%at its level · several runs
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items27.0% ✎▁▁▂▁▃▄█▆▄▅▂▃ beats 13/39very hard · avg 22%▇ 1.2%blunt
BRIDGE clinical text, zero-shot average (87 tasks)41.8% ✎▁▁▁▁▁▂▂▃▄▆█▃ beats 86/97hard · avg 32%▇ 1.1%blunt · at its level
Humanity's Last Exam8.7% ✎▁▁▁▃▆▆▇██▆▁▃ beats 15/104very hard · avg 4.2%▇ 1.1%typical
Multi-IF (aggregation unspecified) lab +176.4% ✎▁▁▁▁▁▁▃▄█▄▁▂ beats 22/41medium · avg 63%▇ 1.1%blunt · several runs
Global PIQA lab80.2% ✎▃▃··▃··▃▃▅█▅ beats 4/11medium · avg 71%▇ 0.9%at its level
MAXIFE lab77.4% ✎▂·▂·▂···▆·▄█ beats 3/11medium · avg 63%▇ 0.9%typical
MultiChallenge lab46.5% ✎▁·▁·▁▃▆▅▃▁█▅ beats 12/29hard · avg 30%▇ 0.8%at its level
VitaBench lab14.1% ✎▄··▃▄▆▆█▄▄▄▂ beats 2/21very hard · avg 10%▇ 0.8%typical
IFBench (aggregation unspecified) lab51.5% ✎▃▇▂▄▅▅▃▄▄▆█▇ beats 37/80medium · avg 39%▇ 0.8%at its level
Creative Writing v3 lab84.4% ✎▂····▂▆··▆█▆ beats 6/14easy · avg 70%▇ 0.7%small run
SimpleQA23.6% ✎▄▄▄▆█▆███·▂█ beats 21/32very hard · avg 6.4%▇ 0.6%typical
KOR-Bench70.2% ✎▄·······█··▄ beats 1/3medium · avg 55%▇ 0.6%at its level
NOVA-63 lab52.5% ✎▃▃·▃···▃█▃▃█ beats 3/11medium · avg 38%▇ 0.6%blunt · at its level
RULER 128k94.5% ✎▃··▅▄▇▅█▇·▃▃ beats 31/32easy · avg 72%▇ 0.4%typical
Chess Puzzles (Epoch)8.0% ✎█▁▁▃▁▂▁▄▂▂▁▁ beats 54/135very hard · avg 5.9%▇ 0.4%sharp · too hard for it · noisy
BeyondAIME56.0% ✎▄·▂·▂▄··▄▄▄█ beats 6/15medium · avg 36%▇ 0.2%at its level
DeepPlanning lab4.9% ✎▂·····▃··▄█▄ beats 1/9very hard · avg 7.1%▇ 0.2%noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)95.2% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 64/140very easy · avg 94%▇ 0.1%blunt · maxed out · noisy
miniF2F (pass@1)5.7% ✎▃·█▅·····▅·▃ beats 0/8very hard · avg 14%▇ 0.1%blunt · noisy · small run
WMT24++ lab69.3% ✎▁·▁▁·▁·▁▃▆█▃ beats 5/27easy · avg 68%▇ 0.1%blunt · at its level

$ where to run it

Through an API

api · Alibaba Cloud Model Studio · $0.75/M · $0.2 in / $2.4 out per 1M tokens · 262k context · www.alibabacloud.com ↗ · price sourceUSD per 1M tokens, Alibaba Cloud Model Studio, Singapore (International deployment scope), qwen3-30b-a3b-thinking-2507 (thinking mode only; output = chain of thought + answer). Chinese-mainland/Global deployments priced differently; free quota, batch and context-cache discounts excluded. Currency: USD. Alibaba Cloud official Model Studio pricing page, observed 2026-09-25.

llama.cpp · recommended

version

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run Qwen3 30B-A3B Thinking 2507 · needs ~18.6 GB · from huggingface.co/lmstudio-community
    chat in the terminal
    llama cli -hf lmstudio-community/Qwen3-30B-A3B-Thinking-2507-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf lmstudio-community/Qwen3-30B-A3B-Thinking-2507-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run Qwen3 30B-A3B Thinking 2507 · needs ~18.6 GB · from huggingface.co/lmstudio-community

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/lmstudio-community/Qwen3-30B-A3B-Thinking-2507-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run Qwen3 30B-A3B Thinking 2507 · needs ~18.8 GB · from ollama.com
    chat in the terminal
    ollama run qwen3:30b

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run Qwen3 30B-A3B Thinking 2507 · needs ~17.2 GB, Mac only · from huggingface.co/lmstudio-community
    chat in the terminal
    mlx_lm.chat --model lmstudio-community/Qwen3-30B-A3B-Thinking-2507-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model lmstudio-community/Qwen3-30B-A3B-Thinking-2507-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Qwen/Qwen3-Next-80B-A3B-Think… 23%, Qwen/Qwen3-30B-A3B-Thinking-2… 22%, ByteDance-Seed/Seed-OSS-36B-I… 18% and 11 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

$ quantization evidence · 12 measurements

How much of the full-precision result a quantized version kept, measured within one study. Rows from the same study share a table and aren't independent.

precisiontestfull → quantizedkept
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)MATH-500 — published row 'Math 500'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 320000.907 → 0.931102.6%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)IFEval — published row 'IfEval (0-shot)'; lm-evaluation-harness task ifeval via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000)0.901 → 0.903100.2%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)GPQA Diamond — published row 'GPQA diamond'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 320000.715 → 0.716100.1%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)MMLU-Pro — published row 'MMLU-Pro (0-shot)'; lm-evaluation-harness task mmlu_pro_chat via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000)0.807 → 0.79198.0%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)AIME 2025 — published row 'AIME 2025'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 320000.867 → 0.83396.1%source ↗
INT4 W4A16 (weight-only INT4, symmetric, group 128; GPTQ, llm-compressor)GSM8K-Platinum — published row 'GSM8k Platinum (0-shot)'; lm-evaluation-harness task gsm8k_platinum_cot_llama via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000)0.856 → 0.72784.9%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)GPQA Diamond — published row 'GPQA diamond'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 320000.715 → 0.724101.3%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)IFEval — published row 'IfEval (0-shot)'; lm-evaluation-harness task ifeval via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000)0.901 → 0.901100.0%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)MMLU-Pro — published row 'MMLU-Pro (0-shot)'; lm-evaluation-harness task mmlu_pro_chat via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000)0.807 → 0.80799.9%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)MATH-500 — published row 'Math 500'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 320000.907 → 0.90699.9%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)GSM8K-Platinum — published row 'GSM8k Platinum (0-shot)'; lm-evaluation-harness task gsm8k_platinum_cot_llama via vLLM OpenAI-compatible server (backend local-chat-completions, chat template, --num_fewshot 0, gen_kwargs do_sample=True,temperature=0.6,top_p=0.95,top_k=20,min_p=0.0,max_gen_toks=64000)0.856 → 0.83898.0%source ↗
INT8 W8A8 (weights INT8 per-channel, activations INT8 per-token dynamic; SmoothQuant+GPTQ, llm-compressor)AIME 2025 — published row 'AIME 2025'; lighteval (litellm endpoint to vLLM server), temperature 0.6, top_p 0.95, top_k 20, max_new_tokens 320000.867 → 0.84297.1%source ↗

$ improve this page

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json