whatmodel.sh / Gemini 3 Pro (Preview)

Gemini 3 Pro (Preview)

+ contribute

Gemini 3 Pro (Preview) is a closed-weights language model from Google, released November 2025.

Catalogue identity notes

Comparison-only. API model 'gemini-3-pro-preview' (MathArena config gemini/riftrunner, date 2025-11-19, thinking included; EQ-Bench 3 entry 'gemini-3-pro-preview'). Superseded by gemini-3-1-pro; not the same checkpoint.

72.6±0.7comparison only: no current run option verifiedfrom 35 benchmark results across 20 sources

$ how it compares

The LLMs ranked either side of Gemini 3 Pro (Preview), out of 828 with a score, then more from Google.

#	model	score	where	RAM	released
62	GPT-5 Pro OpenAI	73±1	api $41.25/M		2025-10
63	LongCat 2.0 (1.6T-A48B) Meituan	73±2	api $1.30/M +local	~2155GB	2026-06
64	Gemini 3 Pro (Preview) Google	73±1	comparison only		2025-11
65	GPT-5 OpenAI	72±1	api $3.44/M		2025-08
66	Qwen3.6 Max Preview Alibaba	72±1	api $2.92/M		2026-04
	more from Google
9	Gemini 4 Argon (limited preview) Google	82±1	limited preview		2026-09
23	Gemini 3.8 Flash Google	77±1	api $1.50/M		2026-09
32	Gemini 3.7 Flash Google	76±1	api $1.50/M		2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #79 of 340 · 35% of its score · 10 tests

Trails MiMo-V2.5 (310B-A15B), Qwen3.6 27B, Muse Glimmer 30B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified	72.9% ✎	▁··▁·▁▂▁▄█▄▁ beats 11/29	hard · avg 19%	▇▇▇▇ 9.1%	sharp
MCP-Atlas (public set) +1	66.6% ✎	▄······▄▄▄▄█ beats 4/6	hard · avg 25%	▇▇▇ 6.4%	at its level · several runs
τ²-bench Telecom lab	98.0% ✎	▁·▅█▆▅▂▃▃▂▃▁ beats 81/89	medium · avg 55%	▇▇ 4.2%	sharp · maxed out · small run
τ³-bench banking_knowledge (v1.0.1 grading)	18.0% ✎	▃▃▂▃·▆▃▂█▅▅▂ beats 6/25	very hard · avg 4.1%	▇ 3.3%	at its level
τ²-bench aggregate	85.4% ✎	▄····█▄▄···▄ beats 2/5	medium · avg 40%	▇ 3.2%	typical
τ³-bench airline (tau2-bench ≥ v1.0.1)	80.5% ✎	▃······▃·▃█▃ beats 1/6	medium · avg 49%	▇ 2.2%	typical
BFCL V4 (December 2025 archive)	68.1% ✎	▁▁▁▂██▅▅▆▂▃▂ beats 72/77	hard · avg 32%	▇ 2.2%	at its level
τ³-bench retail (tau2-bench ≥ v1.0.1)	75.9% ✎	▄▄·█··▄·▄··▄ beats 2/6	medium · avg 46%	▇ 2.0%	typical

2 more tests

test	result	vs the others in its run	difficulty	counts	why
τ³-bench telecom (tau2-bench ≥ v1.0.1)	91.0% ✎	█·▄█▄······▄ beats 3/6	medium · avg 62%	▇ 1.6%	maxed out
τ²-Bench-Verified (GPT-5.1 user)	79.4% ✎	▂··▄···▂▄▄█▄ beats 9/13	medium · avg 45%	▇ 0.8%	typical

coding▁▁▁▃▅▇█▇▅▄▃▁ #72 of 524 · 19% of its score · 3 tests

Beats DeepSeek V4 Flash Vision (Exp), GPT-5.2 here, though they rank higher overall. Trails Gemini 3.5 Flash-Lite, MiMo-V2.5 (310B-A15B), MiMo-V2.5-Pro (1T-A42B) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
SWE-bench Verified	72.9% ✎	▁··▁·▁▂▁▄█▄▁ beats 11/29	hard · avg 19%	▇▇▇▇ 9.1%	sharp
Arena WebDev (Code Arena)	1439.0 ✎	▁▂▄▃▅█▄▇▆▄▁▁ beats 58/119	avg 981	▇▇▇ 8.6%	sharp
WebApp1K-Duo-React pass@1 (onekq-ai)	75.0% ✎	▁··▁·▁▁▁▆█▇█ beats 37/45	hard · avg 32%	▇ 1.7%	typical

chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #16 of 463 * · 4.8% of its score · 1 test

Beats GPT-6 Astra, GPT-6.1 Sol, Claude Sonnet 5.5 here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
LMArena (text)	1485.5 ✎	▁▁▁▂▃▃▃▆▆▇█▃ beats 364/379	avg 1268	▇▇ 4.8%	blunt

math▁▂▃▅▇▅▅█▄▁▁▁ #52 of 462 · 12% of its score · 5 tests

Beats LongCat 2.0 (1.6T-A48B) here, though they rank higher overall. Trails A.X K2 (688B-A33B), DeepSeek V4 Flash (Preview, Apr 2026), GPT-5 here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
MathArena ArXivMath January 2026 (23 problems)	62.0% ✎	▂··▄▂▄█▆▆▄▄█ beats 14/23	hard · avg 22%	▇▇ 4.2%	at its level
MathArena ArXivMath December 2025 (17 problems)	51.1% ✎	▃·▃▃█▁·▄·▁▁▁ beats 14/18	hard · avg 19%	▇ 3.0%	at its level
MATH-Vision	86.6% ✎	█·····▄▄·▄·▄ beats 4/5	medium · avg 46%	▇ 1.8%	typical
OTIS mock AIME 2024–2025	91.4% ✎	▃▃▄▃▃▃▇▇█▃▂▅ beats 149/193	very hard · avg 8.5%	▇ 1.5%	sharp · noisy
FormalProofBench (200 graduate-level Lean 4 problems, agentic)	18.5% ✎	▃▅·▃··▅▃█··▃ beats 9/10	very hard · avg 3.0%	▇ 1.0%	noisy

vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #24 of 182 · 16% of its score · 9 tests

Beats Claude Sonnet 5.5, MiMo-V2.6-Pro, GPT-5.6 Terra here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
CharXiv (reasoning questions) lab	81.4% ✎	▃·▃·▃▃▃·▃▃█▅ beats 6/11	medium · avg 40%	▇▇ 5.6%	typical
Arena vision (style controlled)	1289.3 ✎	▁▁▂▂▂▂▄▅▅▆██ beats 135/145	avg 1068	▇ 3.3%	blunt
MATH-Vision	86.6% ✎	█·····▄▄·▄·▄ beats 4/5	medium · avg 46%	▇ 1.8%	typical
Arena document (style controlled)	1451.1 ✎	▁▁▃▃▄▃▇█▅▇▄▅ beats 12/37	avg 1335	▇ 1.6%	blunt
CharXiv descriptive questions	96.0% ✎	▂▄▄▅▇▄█▇▇▂▁▃ beats 57/58	easy · avg 70%	▇ 1.5%	maxed out
Diagram-MMU diagram question answering (DQA accuracy, scientific TikZ diagrams)	86.5% ✎	▄·····▄▄·█·█ beats 6/6	medium · avg 55%	▇ 0.8%	blunt
Molmo2-VideoCount accuracy (val)	37.1% ✎	▃··▃·▃····█▃ beats 6/6	hard · avg 22%	▇ 0.5%	blunt · at its level
Ai2 Molmo2 human preference Elo (video/image QA, 2026)	1082.0 ✎	▄··▄··▄··▄█▄ beats 4/6	avg 975	▇ 0.4%	blunt

1 more test

test	result	vs the others in its run	difficulty	counts	why
Molmo2-CapTest dense video captioning F1	36.0% ✎	█···█·████·█ beats 1/6	hard · avg 32%	▇ 0.2%	blunt · at its level

web search▂·▂▇▅█▃▄▁▂▁▁ #33 of 92 * · 2.0% of its score · 1 test

Beats Claude Haiku 5.5, Gemini 3.1 Pro (Preview), Claude Opus 4.7 here, though they rank higher overall. Trails GPT-4o mini (2024-07-18), Mistral Small 3.1 24B (2503), GPT-4o (2024-08-06) here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Arena search systems (style controlled)	1201.4 ✎	▁·▁▁▂▂▂█▅▂▁▁ beats 22/30	avg 1055	▇ 2.0%	blunt

other tests7 not in a task · 22% of its score

test	result	vs the others in its run	difficulty	counts	why
Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations)	0.6628 ✎	▁▄▃▃▇▄█▃▅▁▁▃ beats 30/30	avg 0.423	▇▇▇ 7.4%	sharp
GPQA Diamond	92.6% ✎	▁▁▁▁▂▄▆▄▅▇█▅ beats 189/208	hard · avg 40%	▇▇ 4.7%	sharp
Chess Puzzles (Epoch)	31.0% ✎	█▁▁▁▁▂▁▄▃▂▁▁ beats 106/135	very hard · avg 5.9%	▇ 3.3%	sharp · noisy
EQ-Bench 3 (emotional intelligence, judge Elo)	1307.1 ✎	▁·▂▁▄▅▅▃█▄▁▁ beats 48/62	avg 1044	▇ 3.0%	typical
RPC-Bench overall, text input (Z.ai)	32.5% ✎	▄··▄▄·▄▄█·▄▄ beats 6/8	very hard · avg 11%	▇ 2.3%	at its level
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)	64.1% ✎	▁▁·▁▂▂▄█▄▅▄▄ beats 51/66	hard · avg 31%	▇ 1.7%	typical
Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate)	86.4% ✎	▁▂▁▃▅█▂▃▄▃▁▁ beats 15/98	very easy · avg 90%	▇ 0.0%	blunt

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · LMArena leaderboard dataset 20%, Epoch AI — internal evaluatio… 19%, Google DeepMind Gemini model… 10% and 17 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
LMArena leaderboard dataset	5	▇▇▇▇▇▇ 20%	CC BY 4.0
Epoch AI — internal evaluations	4	▇▇▇▇▇▇ 19%	CC BY 4.0
Google DeepMind Gemini model page and model cards (lab-reported scores) · lab-reported	2	▇▇▇ 10%	Factual vendor-reported score (not licensed content)
τ²/τ³-bench leaderboard: Sierra's own evaluation runs	4	▇▇▇ 9%	MIT
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models	1	▇▇ 7%	CC BY 4.0
MathArena ArXivMath / BrokenArXiv / ArXivLean model-output datasets	2	▇▇ 7%	CC BY-SA 4.0
Qwen/Qwen3.5-397B-A17B model card	2	▇▇ 5%	apache-2.0
zai-org/GLM-5 model card	1	▇ 3%	MIT
GLM-5 Team (Zhipu AI) — GLM-5 technical report	1	▇ 3%	CC BY 4.0
EQ-Bench 3 canonical leaderboard results	1	▇ 3%	MIT
RPC-Bench leaderboard (Z.ai research paper comprehension)	1	▇ 2%	Apache-2.0
Berkeley Function Calling Leaderboard result archive	1	▇ 2%	Apache-2.0
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)	1	▇ 2%	Apache-2.0
WebApp1K models leaderboard (onekq-ai)	1	▇ 2%	MIT
Baidu - ERNIE 5.0 Technical Report (arXiv:2602.04705v1)	1	▇ 1%	CC BY 4.0
Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding (Ai2, arXiv 2601.10611v4)	3	▇ 1%	CC BY 4.0
FormalProofBench (Vals AI, arXiv:2603.26996v1)	1	▇ 1%	CC-BY-4.0
Amazon AGI — τ²-Bench-Verified	1	▇ 1%	MIT
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams (arXiv:2608.12262v1)	1	▇ 1%	CC BY 4.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)	1	▇ 0%	Apache-2.0
prior · pull toward the middle		▇ 0%
weighted average		= 72.6