whatmodel.sh / Gemini 3.1 Pro (Preview)

Gemini 3.1 Pro (Preview)

+ contribute

Gemini 3.1 Pro (Preview) is a closed-weights language model from Google, released February 2026.

75.2±0.5#38 of 719 LLMsfrom 63 benchmark results across 29 sources

$ how it compares

The LLMs ranked either side of Gemini 3.1 Pro (Preview), out of 828 with a score, then more from Google.

#	model	score	where	RAM	released
37	Qwen3.8-Flash-Next Alibaba	75±1	local	~59GB	2026-08
38	Qwen3.8-Omni-Flash Alibaba	75±2	api $0.23/M		2026-09
39	Gemini 3.1 Pro (Preview) Google	75±1	api $4.50/M		2026-02
40	Muse Spark 1.2 Meta	75±1	api $2.00/M		2026-08
41	Claude Opus 4.7 Anthropic	75±1	api $10.00/M		2026-04
	more from Google
9	Gemini 4 Argon (limited preview) Google	82±1	limited preview		2026-09
23	Gemini 3.8 Flash Google	77±1	api $1.50/M		2026-09
32	Gemini 3.7 Flash Google	76±1	api $1.50/M		2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #59 of 300 · 14% of its score · 10 tests

Trails MiniMax-M3, GLM-5.1, Claude Opus 4.5 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena Agent (October 2026 aggregate IPS)	-0.0771 IPS ✎	▁▁▁▄▂▃▇█▂▂▁▃ beats 4/49	avg -0.312 IPS	▇▇ 4.0%	sharp
Agents' Last Exam overall full-pass rate (fixed OpenClaw + GUI-as-Tool)	14.1% ✎	▄···█▄▄▄█▄·█ beats 7/10	very hard · avg 1.7%	▇ 1.9%	typical
τ²-bench Telecom lab	99.3% ✎	▁·▅█▆▅▂▃▃▂▃▃ beats 88/89	medium · avg 55%	▇ 1.5%	sharp · maxed out · small run
τ³-bench banking_knowledge (v1.0.1 grading)	26.0% ✎	▃▃▂▃·▆▃▂█▅▅▂ beats 10/25	very hard · avg 4.1%	▇ 1.5%	at its level
Vibe Code Bench (test split, 50 web apps)	32.0% ✎	▃··▃·▃██▃▃▃█ beats 9/14	very hard · avg 2.3%	▇ 1.2%	sharp · at its level · noisy
MCP-Atlas (public set)	69.2% ✎	█··█·██·████ beats 0/7	hard · avg 29%	▇ 1.1%	typical
SWE Atlas (Codebase Q&A, Test Writing, Refactoring; mini-SWE-agent) pass@1	23.7% ✎	▄·▄··█···▄·█ beats 2/6	very hard · avg 6.1%	▇ 1.1%	at its level
Legal Research Bench (agentic, all-pass)	20.8% ✎	▄█··▄▄▄██·▄█ beats 3/12	very hard · avg 5.8%	▇ 0.7%	at its level

2 more tests

test	result	vs the others in its run	difficulty	counts	why
RWSearch	76.3% ✎	████··█····█ beats 0/5	medium · avg 44%	▇ 0.7%	typical
TerminalWorld-Verified (200 tasks) with Terminus-2	55.0% ✎	▄··▄·▄·█···▄ beats 2/5	hard · avg 27%	▇ 0.4%	blunt · at its level

coding▁▁▁▃▆██▇▆▄▃▁ #63 of 458 · 7.1% of its score · 4 tests

Beats DeepSeek V4 Flash Vision (Exp) here, though they rank higher overall. Trails MiMo-V2.5-Pro (1T-A42B), MiniMax-M3, Ornith 1.5 35B-A3B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena WebDev (Code Arena)	1446.2 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 60/119	avg 981	▇ 3.5%	sharp
Luau Leetcode fully solved (Torpedo Software)	82.7% ✎	▁▁▁▁·▁▂▃▅▃█▇ beats 40/49	hard · avg 18%	▇ 1.4%	at its level
Vibe Code Bench (test split, 50 web apps)	32.0% ✎	▃··▃·▃██▃▃▃█ beats 9/14	very hard · avg 2.3%	▇ 1.2%	sharp · at its level · noisy
SWE Atlas (Codebase Q&A, Test Writing, Refactoring; mini-SWE-agent) pass@1	23.7% ✎	▄·▄··█···▄·█ beats 2/6	very hard · avg 6.1%	▇ 1.1%	at its level

chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #14 of 393 * · 2.2% of its score · 2 tests

Beats GPT-6 Astra, GPT-6.1 Sol, Claude Sonnet 5.5 here, though they rank higher overall. Trails Claude Opus 4.6, Muse Spark 1.1, Claude Opus 4.7 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1487.0 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 365/379	avg 1268	▇ 2.0%	blunt
IFEval (aggregation unspecified)	96.1% ✎	█·██·█···█·█ beats 5/5	easy · avg 74%	▇ 0.3%	maxed out

math▁▂▃▅▆▅▄█▄▁▁▁ #29 of 398 · 46% of its score · 20 tests

Beats DeepSeek V4.1 Flash, Gemini 3.8 Flash, DeepSeek V4 Pro (0813) here, though they rank higher overall. Trails Qwen3.7 Max here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
FrontierMath tiers 1–3 v2	59.6% ✎	▂··▁▁▃▄▄█▅▃▃ beats 47/82	very hard · avg 1.2%	▇▇▇▇ 9.1%	sharp · at its level
FrontierMath tier 4 v2	26.8% ✎	▁▂▁▂█▅▂▂▁▁▁▁ beats 31/63	very hard · avg 0.1%	▇▇▇ 8.0%	sharp · at its level
MathArena ArXivMath June 2026 (48 problems)	66.7% ✎	▂···▄·██▆▆▄▂ beats 9/19	very hard · avg 4.6%	▇▇ 5.2%	sharp · at its level
MathArena ArXivMath April 2026 (40 problems)	63.7% ✎	▁·····▅██▄·▁ beats 15/20	very hard · avg 7.2%	▇▇ 4.4%	sharp · at its level
MathArena ArXivMath February 2026 (32 problems)	62.5% ✎	▁▁▁▁█▇▄▃▁▃·▃ beats 22/24	very hard · avg 9.1%	▇ 2.6%	at its level
MathArena ArXivMath March 2026 (30 problems)	68.3% ✎	█▃▅·█▃▃·▅·▃▃ beats 12/14	very hard · avg 15%	▇ 2.2%	at its level
MathArena BrokenArXiv April 2026 (61 problems)	25.8% ✎	▂▂▆█▃▃▂▂▂··▂ beats 10/18	very hard · avg 0.7%	▇ 1.8%	sharp · at its level · noisy
MathArena ArXivMath January 2026 (23 problems)	70.7% ✎	▂··▄▂▄█▆▆▄▄█ beats 18/23	hard · avg 22%	▇ 1.6%	at its level

12 more tests

test	result	vs the others in its run	difficulty	counts	why
MathArena ArXivMath May 2026 (40 problems)	51.7% ✎	▁··▁▄█▄▁·▃·▁ beats 5/17	very hard · avg 10%	▇ 1.6%	at its level
MathArena BrokenArXiv June 2026 (54 problems)	17.6% ✎	▄··█▃▂▂▆▄··▂ beats 6/14	very hard · avg 1.1%	▇ 1.2%	sharp · at its level · noisy
MathArena ArXivMath December 2025 (17 problems)	66.2% ✎	▃·▃▃█▁·▄·▁▁▃ beats 18/18	hard · avg 19%	▇ 1.2%	at its level
MathArena ArXivLean June 2026 (46 problems)	10.9% ✎	▄····▄▄███·▄ beats 2/9	very hard · avg 0.7%	▇ 1.2%	sharp
MathArena BrokenArXiv May 2026 (50 problems)	17.0% ✎	██·▄▄▄····▄▄ beats 4/8	very hard · avg 1.6%	▇ 1.1%	sharp
MathArena BrokenArXiv March 2026 (56 problems)	13.8% ✎	█·▆█▂·▂▂···▂ beats 7/14	very hard · avg 0.8%	▇ 1.1%	sharp · noisy
MathArena BrokenArXiv February 2026 (31 problems)	18.5% ✎	▃▂▂▃█▃▂·▃··▂ beats 12/16	very hard · avg 0.7%	▇ 1.0%	sharp · noisy
MathArena ArXivLean March 2026 (41 problems)	14.6% ✎	▂····▂·····█ beats 2/5	very hard · avg 1.0%	▇ 0.8%	typical
IMO-AnswerBench	90.0% ✎	▁▁·▁▁▁▄▅█▄▃▃ beats 38/42	hard · avg 25%	▇ 0.7%	noisy
MathNet-Solve-Test micro-average accuracy (6,400 olympiad problems)	78.4% ✎	▃▃···█▃·▃·▃▃ beats 8/8	hard · avg 15%	▇ 0.7%	at its level
MathConstraint full release, no tools (329 instances)	51.7% ✎	▄··▄▄·▄▄█▄·▄ beats 7/8	very hard · avg 11%	▇ 0.5%	at its level
OTIS mock AIME 2024–2025	95.6% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 164/193	very hard · avg 8.5%	▇ 0.4%	sharp · maxed out · noisy

vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #19 of 147 · 6.0% of its score · 7 tests

Beats Claude Sonnet 5.5, GPT-5.5, MiMo-V2.6-Pro here, though they rank higher overall. Trails Gemini 3.5 Flash, Claude Opus 4.6, Claude Opus 4.7 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv (reasoning questions) lab	83.3% ✎	▃·▃·▃▃▃·▃▃█▅ beats 7/11	medium · avg 40%	▇ 2.0%	typical
Arena vision (style controlled)	1279.3 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 124/145	avg 1068	▇ 1.4%	blunt
ChartArena chart parsing (English, mAP_high average)	59.2% ✎	▁·▁·▃▃▁▇▃█·▄ beats 22/22	hard · avg 27%	▇ 1.1%	blunt · at its level
Arena document (style controlled)	1458.9 ✎	▁▁▃▃▄▃▇█▅▇▄▅ beats 16/37	avg 1335	▇ 0.7%	blunt
BEAR-Bench (bilingual RU/EN enterprise and academic document images, overall accuracy)	75.1% ✎	▃▃·▅·▃·█·█▅▅ beats 13/14	hard · avg 24%	▇ 0.6%	at its level
Diagram-MMU diagram question answering (DQA accuracy, scientific TikZ diagrams)	86.3% ✎	▄·····▄▄·█·█ beats 5/6	medium · avg 55%	▇ 0.3%	blunt
PureDocBench document parsing (Avg3 over Clean/Digital/Real)	70.4% ✎	▂·▁·▂▂▅█·▁▂▂ beats 16/22	medium · avg 62%	▇ 0.0%	blunt · at its level

web search▂▁▂▆▄█▃▃▁▁·▁ #50 of 80 · 2.3% of its score · 3 tests

Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), Llama 4 Scout (109B-A17B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena search systems (style controlled)	1207.7 ✎	▁·▁▁▂▂▂█▅▂▁▁ beats 25/30	avg 1055	▇ 0.8%	blunt
Legal Research Bench (agentic, all-pass)	20.8% ✎	▄█··▄▄▄██·▄█ beats 3/12	very hard · avg 5.8%	▇ 0.7%	at its level
RWSearch	76.3% ✎	████··█····█ beats 0/5	medium · avg 44%	▇ 0.7%	typical

other tests21 not in a task · 26% of its score

test	result	vs the others in its run	difficulty	counts	why
EBR-bench (Epoch)	14.3% ✎	▄▂·▆█▂▆▄▆▆·▄ beats 8/23	very hard · avg 0.3%	▇ 3.7%	sharp
Furniture Assembly (Epoch)	26.7% ✎	█▆▆▆▆▂▅▂▂▂·▅ beats 5/30	very hard · avg 1.2%	▇ 3.3%	sharp · at its level
Mystery Game Puzzles (Epoch)	34.0% ✎	█··▂▃▇▄█▂▃▁▁ beats 54/74	very hard · avg 9.4%	▇ 2.6%	sharp · noisy
Chess Puzzles (Epoch)	49.0% ✎	█▁▁▁▁▂▁▄▂▂▃▁ beats 128/135	very hard · avg 5.9%	▇ 1.7%	sharp · at its level · noisy
KINA (Knowledge Index of Noah's Ark) accuracy, 899 ten-option items	53.2% ✎	▁▁▂▁▃▄█▆▄▅▂▃ beats 39/39	very hard · avg 22%	▇ 1.5%	blunt · at its level
GPQA Diamond	94.4% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 202/208	hard · avg 40%	▇ 1.5%	sharp
EQ-Bench 3 (emotional intelligence, judge Elo)	1301.8 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 45/62	avg 1044	▇ 1.2%	typical
MMLU-Pro	91.1% ✎	▃▂▅█▃·█▃▅··▃ beats 23/23	medium · avg 57%	▇ 1.2%	typical

13 more tests

test	result	vs the others in its run	difficulty	counts	why
WildClawBench	40.8% ✎	▃·▅▅█·▅█▅▃▃▃ beats 9/17	very hard · avg 11%	▇ 1.1%	at its level
The Token Games Elo (10-model puzzle-duel tournament)	1129.0 ✎	█··▄█··▄·▄·▄ beats 6/7	avg 815	▇ 1.1%	typical
AgentWorldBench (open-ended, overall)	54.6% ✎	▄▄··▄▄·▄█▄██ beats 5/11	hard · avg 25%	▇ 1.1%	blunt · at its level
Legal Research Bench (single-shot, no tools, all-pass)	6.5% ✎	▃▃█▃·▅···█▃▃ beats 9/12	very hard · avg 0.8%	▇ 0.9%	too hard for it
SimpleQA Verified	73.5% ✎	▁▁▃▂▃▃▆▆█▃▂▅ beats 76/78	very hard · avg 8.0%	▇ 0.8%	at its level
FORTE (AGI-Eval)	70.3% ✎	▄···█·····▄█ beats 0/5	medium · avg 37%	▇ 0.8%	at its level
WritingBench	83.7% ✎	▁·▁▂▁··▅█▂▂▁ beats 7/20	medium · avg 55%	▇ 0.7%	typical
Icelandic LLM Leaderboard average (Miðeind, Sept 2026 suite)	66.5% ✎	▁▁▃▂▂▄▁▃▄▄▄█ beats 33/38	very hard · avg 14%	▇ 0.7%	at its level
FinSheet-Bench accuracy (501 questions, 24 spreadsheets)	82.4% ✎	█···██·····█ beats 3/3	medium · avg 52%	▇ 0.5%	blunt
RobloxQA v2.0 multiple choice (Torpedo Software)	96.5% ✎	▁▁▃▁█▄▃▃▆▂▂▂ beats 36/49	easy · avg 78%	▇ 0.5%	maxed out
DiagnosticIQ industrial maintenance MCQ (macro accuracy, 4 options)	72.7% ✎	▁··▃▁▁▁▅█▅▁▅ beats 22/24	medium · avg 52%	▇ 0.4%	blunt · at its level
MirrorCode (Epoch)	8.9% ✎	█·▄▄·▄·▄·▄·█ beats 0/8	very hard · avg 1.5%	▇ 0.4%	sharp · noisy
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	89.6% ✎	▁▂▁▂▅█▂▃▄▃▁▁ beats 42/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

Through an API

api · $4.50/M · $2 in / $12 out per 1M tokens · 1M context · ai.google.dev ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

$ how the score was worked out · Epoch AI — internal evaluatio… 31%, MathArena ArXivMath / BrokenA… 27%, LMArena leaderboard dataset 12% and 26 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Epoch AI — internal evaluations	10	▇▇▇▇▇▇▇▇▇ 31%	CC BY 4.0
MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets	14	▇▇▇▇▇▇▇▇ 27%	CC BY-SA 4.0
LMArena leaderboard dataset	6	▇▇▇▇ 12%	CC BY 4.0
Google DeepMind Gemini model page and model cards (lab-reported scores) · lab-reported	2	▇ 4%	Factual vendor-reported score (not licensed content)
meituan-longcat/LongCat-2.0 model card	5	▇ 3%	mit
Agents' Last Exam — authors' own fixed-harness model comparison	1	▇ 2%	CC BY 4.0
Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode	2	▇ 2%	MIT
Legal Research Bench (Vals AI, arXiv:2610.00609v1)	2	▇ 2%	CC-BY-4.0
Knowledge Index of Noah's Ark (arXiv:2606.05104v2)	1	▇ 2%	CC BY 4.0
τ²/τ³-bench leaderboard: Sierra's own evaluation runs	1	▇ 1%	MIT
EQ-Bench 3 canonical leaderboard results	1	▇ 1%	MIT
Vibe Code Bench (Vals AI, arXiv:2603.04601v3)	1	▇ 1%	CC-BY-4.0
MMLU-Pro author-released evaluation runs	1	▇ 1%	Apache-2.0
WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation (arXiv:2605.10912v1)	1	▇ 1%	CC BY 4.0
The Token Games: Evaluating Language Model Reasoning with Puzzle Duels — authors' own tournament	1	▇ 1%	CC BY-SA 4.0
zai-org/GLM-5.2 model card	1	▇ 1%	mit
Qwen/Qwen-AgentWorld-35B-A3B model card	1	▇ 1%	Apache-2.0
SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution (arXiv:2605.08366v1)	1	▇ 1%	CC BY 4.0
ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats (arXiv:2606.01348v3)	1	▇ 1%	CC BY 4.0
Miðeind Icelandic LLM Leaderboard	1	▇ 1%	CC BY 4.0
MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval — authors' own model runs	1	▇ 1%	CC BY 4.0
BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models (arXiv:2608.17895v1)	1	▇ 1%	CC BY 4.0
MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs — authors' own model runs	1	▇ 0%	CC BY 4.0
FinSheet-Bench: From Simple Lookups to Complex Reasoning, Where LLMs Break on Financial Spreadsheets — authors' own model runs	1	▇ 0%	CC BY 4.0
TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks — authors' own Terminus-2 model comparison	1	▇ 0%	CC BY 4.0
DiagnosticIQ: A Benchmark for LLM-Based Industrial Maintenance Action Recommendation from Symbolic Rules (arXiv:2605.08614v1)	1	▇ 0%	CC BY 4.0
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams (arXiv:2608.12262v1)	1	▇ 0%	CC BY 4.0
How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings (arXiv:2605.07492v2)	1	▇ 0%	CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 75.2