GPT-5.1

+ contribute

GPT-5.1 is a closed-weights language model from OpenAI, released November 2025.

70.9±0.8#75 of 719 LLMsfrom 28 benchmark results across 13 sources

$ how it compares

The LLMs ranked either side of GPT-5.1, out of 828 with a score, then more from OpenAI.

#	model	score	where	RAM	released
78	Ling 3.0 Flash (124B-A5B) inclusionAI (Ant Group)	71±2	api $0.090/M +local	~75GB	2026-08
79	Mistral Large 4 (public preview) Mistral	71±2	api $2.06/M		2026-10
80	GPT-5.1 OpenAI	71±1	api $3.44/M		2025-11
81	GLM-5.1 Z.ai	71±1	api $2.15/M +local	~119GB	2026-04
82	Qwen3.5 397B-A17B Alibaba	71±0	api $1.09/M +local	~65GB	2026-02
	more from OpenAI
1	GPT-6 Astra OpenAI	87±1	api $20.00/M		2026-09
3	GPT-6.1 Sol OpenAI	85±1	api $4.00/M		2026-09
7	GPT-6 Sol OpenAI	83±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #92 of 300 · 26% of its score · 7 tests

Beats Grok 4.20 (0309 reasoning) here, though they rank higher overall. Trails Hy3 preview (295B-A21B), Qwen3.6 35B-A3B, MiniMax-M2.7 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified	68.0% ✎	▁··▁·▁▂▃▄█▄▁ beats 7/29	hard · avg 19%	▇▇▇▇ 11%	sharp · at its level
ContractBench success rate (33 observation-contract tool tasks)	48.5% ✎	▆··▆▄▂▄▂▆▆█▄ beats 12/23	very hard · avg 14%	▇▇ 4.6%	sharp · at its level
τ²-bench Telecom lab +1	95.6% ✎	▃·▅▅▃▃··█▃▃▅ beats 10/13	very hard · avg 13%	▇▇ 4.4%	sharp · at its level · noisy · several runs
τ²-bench Retail	78.1% ✎	▁····▂▁▃▆█▃▁ beats 39/44	medium · avg 38%	▇ 2.0%	typical
τ²-bench Airline	64.0% ✎	▁▁·▂▁▃▂▁█▅▃▃ beats 36/42	hard · avg 29%	▇ 1.9%	at its level
KlinikeBench strict pass@1 (333 interactive clinical encounters, Terminus-2)	15.3% ✎	▁▁·▁▁▁▁▅▃█▅█ beats 13/27	very hard · avg 2.4%	▇ 1.2%	typical
τ²-Bench-Verified (GPT-5.1 user)	76.8% ✎	▂··▄···▂▄▄█▄ beats 7/13	medium · avg 45%	▇ 0.9%	typical

coding▁▁▁▃▆██▇▆▄▃▁ #91 of 458 · 22% of its score · 3 tests

Beats Grok 4.20 (0309 reasoning) here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Qwen3 235B-A22B Thinking 2507, Qwen3.6 35B-A3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified	68.0% ✎	▁··▁·▁▂▃▄█▄▁ beats 7/29	hard · avg 19%	▇▇▇▇ 11%	sharp · at its level
Arena WebDev (Code Arena)	1394.8 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 42/119	avg 981	▇▇▇▇ 9.4%	sharp
WebApp1K-Duo-React pass@1 (onekq-ai)	76.5% ✎	▁··▁·▁▁▁▆█▇█ beats 42/45	hard · avg 32%	▇ 2.0%	typical

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #46 of 393 * · 5.3% of its score · 1 test

Beats MiMo-V2.6-Flash (309B-A15B), GPT-6 Luna, GPT-5.6 Luna here, though they rank higher overall. Trails Claude Sonnet 4.5, MiMo-V2.5-Pro (1T-A42B), GLM-5 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1455.8 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 324/379	avg 1268	▇▇ 5.3%	blunt

math▁▂▃▅▆▅▄█▄▁▁▁ #74 of 398 * · 2.2% of its score · 1 test

Beats Claude Sonnet 4.6, Claude Opus 4.5, Qwen3.7 Plus here, though they rank higher overall. Trails Granite 4.2 30B, INTELLECT-3, EXAONE 4.5 33B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
OTIS mock AIME 2024–2025	88.6% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 138/193	very hard · avg 8.5%	▇ 2.2%	sharp · noisy

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #49 of 147 · 8.2% of its score · 3 tests

Beats MiMo-V2.6-Pro, DeepSeek V4 Flash Vision (Exp), GPT-5 here, though they rank higher overall. Trails Qwen3.6 Flash, Gemma 4 31B, Gemini 3.5 Flash-Lite here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena vision (style controlled)	1251.4 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 104/145	avg 1068	▇ 3.7%	blunt
CharXiv (reasoning questions) lab	67.0% ✎	▄··▂█··▃···▄ beats 7/9	hard · avg 34%	▇ 2.8%	at its level
Arena document (style controlled)	1415.8 ✎	▁▃▃▃▄▃▇█▅▇▄▅ beats 1/37	avg 1335	▇ 1.8%	blunt

web search▂▁▂▆▄█▃▃▁▁·▁ #40 of 80 * · 2.2% of its score · 1 test

Beats Gemini 3.1 Pro (Preview), Qwen3.7 Max, Gemini 3.5 Flash here, though they rank higher overall. Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), Llama 4 Scout (109B-A17B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena search systems (style controlled)	1183.2 ✎	▁·▁▁▂▂▂█▅▂▁▁ beats 11/30	avg 1055	▇ 2.2%	blunt

other tests12 not in a task · 44% of its score

test	result	vs the others in its run	difficulty	counts	why
KMMLU-HARD (Daekeun Kim, zero-shot)	72.8% ✎	█···▁▁▁▂▃·▆▂ beats 16/23	very hard · avg 29%	▇▇▇ 7.5%	sharp · at its level
GPQA Diamond	87.6% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 158/208	hard · avg 40%	▇▇ 6.0%	sharp
ARC-AGI-2 public evaluation set (ARC Prize model_baseline, pass@2)	18.3% ✎	█··▂▅▁▃▁▃·▁▃ beats 16/24	very hard · avg 0.3%	▇▇ 4.9%	sharp · noisy
Mystery Game Puzzles (Epoch)	16.0% ✎	█··▂▃▇▄█▂▃▁▁ beats 24/74	very hard · avg 9.4%	▇▇ 3.9%	sharp · noisy
ARC-AGI-1 public evaluation set (ARC Prize model_baseline, pass@2)	77.1% ✎	▁·▇▃▁·█▇▁·▇▃ beats 19/27	very hard · avg 4.1%	▇ 3.7%	sharp · at its level · noisy
Medmarks-V weighted mean win rate (61 models, 71 configurations)	63.9% ✎	▂▃▅▂▆▃█▅█▅▃▆ beats 34/34	medium · avg 42%	▇ 3.7%	blunt · at its level
CLIcK Korean (Daekeun Kim)	89.8% ✎	▆▄▄▄█▄▂·▄█·▄ beats 16/23	medium · avg 58%	▇ 3.6%	typical
EQ-Bench 3 (emotional intelligence, judge Elo)	1314.4 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 54/62	avg 1044	▇ 3.3%	typical

4 more tests

test	result	vs the others in its run	difficulty	counts	why
Chess Puzzles (Epoch)	32.0% ✎	█▁▁▁▁▂▁▄▃▂▁▁ beats 108/135	very hard · avg 5.9%	▇ 3.2%	sharp · noisy
HAE-RAE 1.0 (Daekeun Kim)	90.6% ✎	▄▄▄▂▂█▂▄▃█▂▄ beats 15/22	medium · avg 63%	▇ 2.7%	typical
SimpleQA Verified	48.0% ✎	▁▁▃▂▃▃▆▆█▃▂▅ beats 49/78	very hard · avg 8.0%	▇ 2.0%	at its level
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	87.9% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 22/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · $3.44/M · $1.25 in / $10 out per 1M tokens · 400k context · platform.openai.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

$ how the score was worked out · Epoch AI — internal evaluatio… 28%, LMArena leaderboard dataset 22%, Daekeun Kim Korean LLM benchm… 14% and 10 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Epoch AI — internal evaluations	6	▇▇▇▇▇▇▇▇ 28%	CC BY 4.0
LMArena leaderboard dataset	5	▇▇▇▇▇▇▇ 22%	CC BY 4.0
Daekeun Kim Korean LLM benchmark runs	3	▇▇▇▇ 14%	MIT
ARC Prize Foundation model_baseline outputs on ARC-AGI-1/2 public evaluation sets	2	▇▇▇ 9%	MIT
HyperCLOVA X 32B Think (NAVER Cloud, arXiv:2601.03286v1)	3	▇▇ 6%	CC-BY-4.0
OpenAI vendor-reported benchmark scores for its own models (openai.com announcement posts and Deployment Safety Hub system cards) · lab-reported	2	▇ 5%	Factual vendor-reported score (not licensed content)
ContractBench: Can LLM Agents Preserve Observation Contracts? (arXiv:2605.17281v2)	1	▇ 5%	CC BY 4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models	1	▇ 4%	CC BY 4.0
EQ-Bench 3 canonical leaderboard results	1	▇ 3%	MIT
WebApp1K models leaderboard (onekq-ai)	1	▇ 2%	MIT
KlinikeBench: Evaluating Language Models Beyond Diagnostic Accuracy (arXiv:2609.38480v2)	1	▇ 1%	CC BY 4.0
Amazon AGI — τ²-Bench-Verified	1	▇ 1%	MIT
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 70.9