whatmodel.sh / Gemini 2.5 Pro Preview (05-06)
Gemini 2.5 Pro Preview (05-06)
↗ share
+ contribute
Gemini 2.5 Pro Preview (05-06) is a closed-weights language model from Google, released May 2025. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2025-05-06). Preview API model gemini-2.5-pro-preview-05-06. Gemini API deprecations page: shutdown date December 2, 2025.
$ how it compares
The LLMs ranked either side of Gemini 2.5 Pro Preview (05-06), out of 828 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 135 |  Qwen3-Coder-Next (80B-A3B) | 67±2 | api $0.60/M +local | ||
| 136 | Gemini 2.5 Pro Preview (03-25) | 67±2 | comparison only | ||
| 137 | Gemini 2.5 Pro Preview (05-06) | 67±1 | comparison only | ||
| 138 |  GigaChat 3.5 Ultra Reasoning (432B-A28B) | 67±1 | local | ||
| 139 | Qwen3.5 35B-A3B | 67±0 | api $0.35/M +local | ||
| more from Google | |||||
| 9 | Gemini 4 Argon (limited preview) | 82±1 | limited preview | ||
| 23 | Gemini 3.8 Flash | 77±1 | api $1.50/M | ||
| 32 | Gemini 3.7 Flash | 76±1 | api $1.50/M | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #238 of 340 * · 2.1% of its score · 1 test
Trails SmolLM3 3B, Jamba 1.5 Mini, Granite 4.1 3B here, though they rank lower overall.
coding▁▁▁▃▅▇█▇▅▄▃▁ #104 of 524 · 41% of its score · 3 tests
Beats Grok 4.20 (0309 reasoning), Grok 4.3, Muse Glimmer 30B here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, o4-mini, Qwen3 235B-A22B Thinking 2507 here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) | 76.9% ✎ | ▂·▂▃▅▂▃█▆▄▃▂ beats 44/49 | 28% |
| LiveCodeBench official (Aug 1, 2024–May 1, 2025) | 71.8% ✎ | ▃▃▅▃··▃▅█▅▅▃ beats 12/15 | 7.8% |
| WebApp1K-Duo-React pass@1 (onekq-ai) | 67.4% ✎ | ▁··▁·▁▁▁▆█▇█ beats 25/45 | 4.9% |
math▁▂▃▅▇▅▅█▄▁▁▁ #109 of 462 * · 7.3% of its score · 1 test
Beats Claude Sonnet 4.6, MiniMax-M3, Nemotron 3 Ultra 550B-A55B here, though they rank higher overall. Trails Gemini 2.5 Flash, GLM-4.5-Air, Granite 4.2 30B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MATH level 5 | 95.9% ✎ | ▂▂▅▆▄█▆▃▅▂▃▄ beats 83/92 | 7.3% |
vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #84 of 182 * · 1.9% of its score · 1 test
Beats DeepSeek V4 Flash Vision (Exp), GPT-5.4 nano, GPT-5 mini here, though they rank higher overall. Trails GPT-4.1, Gemma 4 12B, Qwen3.5 9B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MSEarth-Bench Earth-science figure MCQ (Overall accuracy) | 61.3% ✎ | █·██····██·█ beats 5/5 | 1.9% |
other tests13 not in a task · 47% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond | 66.7% ✎ | ▁▁▁▁▂▄▆▄▅▇█▅ beats 99/208 | 17% |
| AHELM Speech Robust Bench (LibriSpeech-clean with noise) transcription WER | 3.9% ✎ | ▂·█··▆▄·▂▂·▄ beats 13/13 | 10% |
| AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge) | 68.4% ✎ | ▁▁·▁▂▂▄█▄▅▄▄ beats 54/66 | 5.1% |
| HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge) | 71.8% ✎ | ▁▁▁▁·▂·█▆█▇▁ beats 21/31 | 3.8% |
| AHELM VoxCeleb2 same-speaker detection (exact match) | 75.1% ✎ | ▂·▂·▂█▂▄▄··▄ beats 12/13 | 3.6% |
| AHELM CoRe-Bench conversational audio reasoning (pseudo-exact match) | 81.3% ✎ | ▂·▄··▂▂·▄▂█▄ beats 13/13 | 2.2% |
| AHELM MUStARD audio sarcasm detection (exact match) | 65.5% ✎ | ▂·······▆█▆▆ beats 12/13 | 2.1% |
| AHELM LibriSpeech transcription by audio LLMs (WER) | 3.9% ✎ | ▃···▃·█▃▅▃▅█ beats 13/13 | 2.1% |
5 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM AIR-Bench Foundation music subsets (exact match) | 68.3% ✎ | ▃█▃▅▃·▃·▃▅▃▃ beats 9/13 | 0.8% |
| AHELM MELD audio emotion classification (pseudo-exact match) | 47.3% ✎ | ▃··▃▃▃·▃▅▅█▅ beats 10/13 | 0.6% |
| AHELM FLEURS 7-language transcription by audio LLMs (WER) | 21.1% ✎ | ▄▂···▂▆▂█▂·▃ beats 13/13 | 0.3% |
| AHELM Multilingual LibriSpeech 7-language transcription by audio LLMs (WER) | 19.8% ✎ | ▂▂▂··▃▂·▂▂█▆ beats 3/13 | 0.0% |
| AHELM VocalSound human-sound recognition (prefix exact match) | 86.0% ✎ | ██▄▄·█·█·█▄▄ beats 12/13 | 0.0% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Aider polyglot benchmark runs 28%, Epoch AI — internal evaluatio… 24%, AHELM: A Holistic Evaluation… 22% and 5 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Aider polyglot benchmark runs | 1 | 28% |
 Epoch AI — internal evaluations | 2 | 24% |
 AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2) | 10 | 22% |
| Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval) | 2 | 9% |
| LiveCodeBench official website results (2025-08-01) | 1 | 8% |
| WebApp1K models leaderboard (onekq-ai) | 1 | 5% |
| Berkeley Function Calling Leaderboard result archive | 1 | 2% |
| MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (arXiv:2505.20740v3) | 1 | 2% |
| prior · pull toward the middle | 1% | |
| weighted average | = 66.7 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json