whatmodel.sh / Qwen3 235B-A22B Thinking 2507

Qwen3 235B-A22B Thinking 2507

+ contribute

Qwen3 235B-A22B Thinking 2507 is an open-weights language model from Alibaba, released July 2025. Its published size is 235B parameters (22B active per token) and a 131k-token context window. Licence: apache-2.0.

66.4±0.5#123 of 719 LLMsfrom 92 benchmark results across 12 sources

$ how it compares

The LLMs ranked either side of Qwen3 235B-A22B Thinking 2507, out of 828 with a score, then more from Alibaba.

#	model	score	where	RAM	released
140	DeepSeek V3.1 Terminus DeepSeek	67±1	local	~230GB	2025-09
141	DeepSeek V3.2 Exp DeepSeek	67±1	local	~726GB	2025-09
142	Qwen3 235B-A22B Thinking 2507 Alibaba	66±1	api $0.97/M +local	~76GB	2025-07
143	Grok 4.1 Fast (Reasoning) SpaceXAI (xAI)	66±1	api $0.28/M		2025-11
144	o4-mini OpenAI	66±1	api $1.93/M		2025-04
	more from Alibaba
18	Qwen3.8 Max (2.4T-A95B) Alibaba	78±1	api $3.00/M +local	~380GB	2026-08
34	Qwen3.8 Max (0902) Alibaba	76±1	api $3.00/M		2026-09
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #145 of 300 · 22% of its score · 8 tests

Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
τ²-bench Telecom lab +3	45.6% ✎	▃·█········█ beats 5/6	very hard · avg 10%	▇▇▇▇ 11%	sharp · at its level · several runs
τ²-bench Retail +2	68.2% ✎	█···█·█·██·█ beats 1/5	medium · avg 36%	▇ 3.4%	at its level · several runs
τ²-bench Airline +2	58.0% ✎	▄·····█··▄·█ beats 2/5	hard · avg 32%	▇ 2.2%	at its level · several runs
τ-bench Retail lab +1	67.8% ✎	▁····▃·▂▇█▂▆ beats 17/28	medium · avg 36%	▇ 2.1%	at its level · several runs
τ-bench Airline lab +1	46.0% ✎	▁▁▁·▅▄▁█▇▁▁▇ beats 15/28	hard · avg 23%	▇ 1.6%	at its level · several runs
τ²-bench aggregate lab	58.5% ✎	▁▁▁·▂▂▄▃▃▅█▂ beats 9/37	hard · avg 33%	▇ 1.1%	at its level
BFCL v3 +2	75.7% ✎	▁▁▁▁▁▃▅█▆▅▄▃ beats 166/170	medium · avg 59%	▇ 0.7%	blunt · at its level · several runs
Galileo Agent Leaderboard v2 (action completion)	34.0% ✎	▅▃·▃▃▅▅█▃█▅▅ beats 6/19	hard · avg 27%	▇ 0.4%	at its level

coding▁▁▁▃▆██▇▆▄▃▁ #88 of 458 · 3.9% of its score · 2 tests

Beats Grok 4.20 (0309 reasoning), GPT-5.1, Grok 4.3 here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LiveCodeBench v6 (Feb–May 2025) lab +2	74.1% ✎	▁▁▃▁█▅▄▇▅▅▃█ beats 26/38	medium · avg 43%	▇ 2.6%	at its level · several runs
LiveCodeBench (version unspecified)	75.4% ✎	▄······▄█▄·▄ beats 3/5	medium · avg 39%	▇ 1.3%	at its level

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #110 of 393 · 7.3% of its score · 4 tests

Beats LongCat 2.0 (1.6T-A48B), Step 3.5 Flash (196B-A11B), GPT-5 mini here, though they rank higher overall. Trails Nemotron-Cascade-8B, Gemini 2.5 Flash, Mistral Large 3 (675B-A41B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena-Hard v2 aggregate lab +1	79.7% ✎	▁▁··▄▃▁▃█▁▁▇ beats 19/22	hard · avg 28%	▇ 3.2%	sharp · at its level · several runs
LMArena (text)	1400.0 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 244/379	avg 1268	▇ 2.1%	blunt
IFEval (aggregation unspecified) lab +2	87.8% ✎	▁▁▃▁▂▄▄▆█▃▂▁ beats 93/141	easy · avg 77%	▇ 1.4%	several runs
IFEval prompt-level strict	89.3% ✎	███··█·█···█ beats 3/5	easy · avg 74%	▇ 0.7%	typical

math▁▂▃▅▆▅▄█▄▁▁▁ #67 of 398 · 18% of its score · 6 tests

Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.7 Plus here, though they rank higher overall. Trails INTELLECT-3 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
HMMT 2025 (edition unspecified) +3	83.8% ✎	▄··▄··▄·▄··█ beats 5/5	hard · avg 29%	▇▇▇ 7.1%	sharp · several runs
AIME 2025 +2	92.5% ✎	▁·▁▁▁▂▅▆█▇▅▁ beats 124/133	medium · avg 51%	▇▇ 5.2%	sharp · several runs
AIME 2024	93.9% ✎	▁▁▁▂▁▃▁▅███▃ beats 83/85	medium · avg 60%	▇ 2.3%	sharp
OTIS mock AIME 2024–2025	86.7% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 131/193	very hard · avg 8.5%	▇ 1.6%	sharp · noisy
PolyMATH lab +1	60.1% ✎	▁···▃▁·▇█▄▄▄ beats 17/23	hard · avg 32%	▇ 1.4%	at its level · several runs
MATH-500	99.6% ✎	█···██·█·█·█ beats 5/5	very easy · avg 91%	▇ 0.6%	maxed out · noisy

other tests24 not in a task · 48% of its score

test	result	vs the others in its run	difficulty	counts	why
GPQA Diamond +2	80.1% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 128/208	hard · avg 43%	▇▇▇▇ 10%	sharp · at its level · several runs
MMLU-Pro +3	84.4% ✎	▄·▄···█·▄··▄ beats 2/5	medium · avg 56%	▇▇ 4.8%	several runs
MMLU-Redux +3	91.4% ✎	████··█····█ beats 4/5	medium · avg 73%	▇▇ 3.9%	several runs
Humanity's Last Exam lab +2	18.2% ✎	▁▁▁▃▆▆▇██▆▁▃ beats 43/104	very hard · avg 5.2%	▇ 3.3%	several runs
GPQA (subset unspecified) lab +1	81.1% ✎	▂▁▂▂▄▃▆▃▅█▄▃ beats 96/104	hard · avg 49%	▇ 3.2%	sharp · several runs
INCLUDE lab +2	81.0% ✎	▂····▂▂▄▄▄█▄ beats 31/36	medium · avg 59%	▇ 2.9%	several runs
OJBench +2	32.1% ✎	▄······▄▄·▄█ beats 1/5	very hard · avg 13%	▇ 2.9%	at its level · several runs
MMLU-ProX lab +2	81.0% ✎	▁·▁▁▁▂▁▃▂█▅▄ beats 26/35	medium · avg 60%	▇ 1.8%	several runs

16 more tests

test	result	vs the others in its run	difficulty	counts	why
SuperGPQA lab +2	64.9% ✎	▂▁▂▂▅▆▃▇▇▇█▃ beats 41/46	medium · avg 39%	▇ 1.8%	at its level · several runs
WritingBench lab +2	88.3% ✎	▁·▁▂▁··▅█▂▂▃ beats 20/20	easy · avg 70%	▇ 1.6%	several runs
Medmarks-V weighted mean win rate (61 models, 71 configurations)	60.3% ✎	▂▃▅▂▆▃█▅█▅▃▆ beats 30/34	medium · avg 42%	▇ 1.5%	blunt · at its level
Multi-IF (aggregation unspecified) lab +2	80.6% ✎	▁▁▁▁▁▁▃▄█▄▁▂ beats 33/41	medium · avg 60%	▇ 1.5%	blunt · several runs
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	32.1% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 25/39	very hard · avg 22%	▇ 1.4%	blunt · at its level
LiveBench 2024-11-25 lab +2	78.4% ✎	▄··▂·▂·▂█▆▃▂ beats 12/13	medium · avg 60%	▇ 1.4%	several runs
Creative Writing v3 lab +1	86.1% ✎	▂····▂▆··▆█▆ beats 11/14	easy · avg 67%	▇ 1.1%	small run · several runs
ZebraLogic	97.5% ✎	██··█·██···█ beats 5/5	easy · avg 75%	▇ 1.1%	maxed out
VitaBench	21.5% ✎	█····██·██·█ beats 1/5	very hard · avg 7.8%	▇ 1.0%	at its level
Chess Puzzles (Epoch)	12.0% ✎	█▁▁▁▁▃▁▄▂▂▁▁ beats 57/135	very hard · avg 5.9%	▇ 0.8%	sharp · noisy
Mystery Game Puzzles (Epoch)	9.0% ✎	█··▂▃▇▄█▂▃▁▁ beats 5/74	very hard · avg 9.4%	▇ 0.7%	sharp · too hard for it · noisy
SimpleQA Verified	40.4% ✎	▁▁▃▂▃▃▆▆█▃▂▅ beats 34/78	very hard · avg 8.0%	▇ 0.7%	at its level
BeyondAIME	71.5% ✎	█··▄····▄··█ beats 4/5	medium · avg 41%	▇ 0.6%	at its level
SOOHAK Challenge (340 problems, Avg@3)	8.0% ✎	█·█████··█·█ beats 1/7	very hard · avg 4.1%	▇ 0.3%	typical
SOOHAK-Mini (702 problems, Avg@3)	56.2% ✎	▄··▄·▄·▄▄█·▄ beats 2/7	medium · avg 36%	▇ 0.3%	blunt · at its level
miniF2F (pass@1)	11.9% ✎	██······▄··▄ beats 1/5	very hard · avg 13%	▇ 0.1%	blunt · at its level · noisy

$ where to run it

Through an API

api · Novita AI · $0.97/M · $0.3 in / $3 out per 1M tokens · 131k context · novita.ai ↗ · price sourceUSD per 1M text tokens, standard uncached input and output. Model endpoint qwen/qwen3-235b-a22b-thinking-2507. Product: Serverless chat completions. API model: qwen/qwen3-235b-a22b-thinking-2507. Settings: service tier=standard, batch=no, cache=uncached input. Additional/excluded charges: Tools, cache storage, priority service and other modalities excluded.. Currency: USD.

llama.cpp · recommended

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run Qwen3 235B-A22B Thinking 2507 · needs ~71 GB · from huggingface.co/unsloth
chat in the terminal
llama cli -hf unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF:UD-Q2_K_XL
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF:UD-Q2_K_XL

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run Qwen3 235B-A22B Thinking 2507 · needs ~132.2 GB, Mac only · from huggingface.co/lmstudio-community
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run Qwen3 235B-A22B Thinking 2507 · needs ~141.5 GB · from ollama.com
chat in the terminal
ollama run qwen3:235b-a22b-thinking-2507-q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run Qwen3 235B-A22B Thinking 2507 · needs ~132.2 GB, Mac only · from huggingface.co/lmstudio-community
chat in the terminal
mlx_lm.chat --model lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model lmstudio-community/Qwen3-235B-A22B-Thinking-2507-MLX-4bit

Official weights and documentation: huggingface.co/Qwen ↗

$ how the score was worked out · Meituan LongCat-Flash-Thinkin… 23%, Qwen/Qwen3-235B-A22B-Thinking… 21%, Qwen/Qwen3-Next-80B-A3B-Think… 20% and 9 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Meituan LongCat-Flash-Thinking model card (evaluation table)	18	▇▇▇▇▇▇▇ 23%	MIT
Qwen/Qwen3-235B-A22B-Thinking-2507 model card · lab-reported	24	▇▇▇▇▇▇ 21%	apache-2.0
Qwen/Qwen3-Next-80B-A3B-Thinking model card · lab-reported	19	▇▇▇▇▇▇ 20%	apache-2.0
Qwen3-VL-235B-A22B-Thinking model card · lab-reported	14	▇▇▇ 10%	Apache-2.0
Qwen/Qwen3.5-27B model card · lab-reported	5	▇▇ 8%	apache-2.0
Epoch AI — internal evaluations	5	▇▇ 7%	CC BY 4.0
Qwen (Alibaba) vendor-reported benchmark scores for its own models (qwen.ai launch posts, Qwen2.5-VL and Qwen3-VL technical reports, Qwen3.8-Flash-Next card) · lab-reported	1	▇ 4%	Factual vendor-reported score (not licensed content)
LMArena leaderboard dataset	1	▇ 2%	CC BY 4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models	1	▇ 1%	CC BY 4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 1%	CC BY 4.0
Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs — authors' own model runs	2	▇ 1%	CC BY-SA 4.0
Galileo Agent Leaderboard (v1 and v2 results)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 66.4