whatmodel.sh / Gemini 2.0 Pro Experimental (02-05)

Gemini 2.0 Pro Experimental (02-05)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Gemini 2.0 Pro Experimental (02-05) is a closed-weights language model from Google, released February 2025. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2025-02-05). Experimental API model gemini-2.0-pro-exp-02-05. Listed as shut down in the Gemini API changelog.

60.7±1.3comparison only: no current run option verifiedfrom 4 benchmark results across 3 sources

$ how it compares

The LLMs ranked either side of Gemini 2.0 Pro Experimental (02-05), out of 828 with a score, then more from Google.

#modelscorewhereRAMreleased
222Devstral Small 2 24B Mistral61±3local~17GB2025-12
223o1-mini OpenAI61±1comparison only2024-09
224Gemini 2.0 Pro Experimental (02-05) Google61±1comparison only2025-02
225Qwen3 30B-A3B Thinking 2507 Alibaba61±1api $0.75/M +local~20GB2025-07
226Qwen3.5 4B Alibaba61±1local~4.3GB2026-03
more from Google
9Gemini 4 Argon (limited preview) Google82±1limited preview2026-09
23Gemini 3.8 Flash Google77±1api $1.50/M2026-09
32Gemini 3.7 Flash Google76±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▅▇█▇▅▄▃▁ #199 of 524 * · 45% of its score · 1 test

Beats Qwen3.5 9B, Devstral 2 (2512), Qwen3-VL 32B Thinking here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 8B Reasoning 2512, Phi-4-reasoning 14B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Aider polyglot (225 exercises, two attempts)35.6% ✎▂·▂▃▅▂▃█▆▄▂▂ beats 18/49very hard · avg 8.5%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 45%sharp · at its level
math▁▂▃▅▇▅▅█▄▁▁▁ #199 of 462 * · 25% of its score · 1 test

Beats MiniMax-M2.1, GPT-4.1 here, though they rank higher overall. Trails Phi-4-mini-reasoning 3.8B, OpenThinker3 7B, Olmo 3 7B Think here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
MATH level 583.5% ✎▂▂▅▆▄█▆▃▅▂▃▄ beats 70/92hard · avg 32%▇▇▇▇▇▇▇▇▇▇ 25%sharp
other tests2 not in a task · 29% of its score
testresultvs the others in its rundifficultycountswhy
GPQA Diamond65.7% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 96/208hard · avg 40%▇▇▇▇▇▇▇▇▇▇▇ 29%sharp · at its level
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)99.2% ✎▁▁▂▃▄▄█▆▃▂▂▃ beats 139/140very easy · avg 94%▇ 0.6%blunt · maxed out · noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Epoch AI — internal evaluatio… 54%, Aider polyglot benchmark runs 45%, Vectara hallucination leaderb… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Epoch AI — internal evaluations2▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 54%CC BY 4.0
Aider polyglot benchmark runs1▇▇▇▇▇▇▇▇▇▇▇▇▇ 45%Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)1▇ 1%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 60.7

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json