whatmodel.sh / Gemini 2.5 Pro Preview (03-25)
Gemini 2.5 Pro Preview (03-25)
↗ share
+ contribute
Gemini 2.5 Pro Preview (03-25) is a closed-weights language model from Google, released April 2025. Licence: proprietary.
Catalogue identity notes
Comparison-only, proposed so the Galileo Agent Leaderboard v1 (galileo-agent-v1-tsq, 28 catalogue models) and WebApp1K-Duo (webapp1k-duo-react-pass1, 43 catalogue models) rows count; also its aider polyglot run of 2025-04-12 (aider-polyglot-runs). Epoch's math_level_5.csv 'gemini-2.5-pro-preview-03-25' run started 2025-05-07, after the repoint, so it is not mapped. Paid preview API model gemini-2.5-pro-preview-03-25: Gemini API changelog, April 4, 2025: 'Released gemini-2.5-pro-preview-03-25, a public preview Gemini 2.5 Pro version with billing enabled. You can continue to use gemini-2.5-pro-exp-03-25 on the free tier.' May 6, 2025: 'gemini-2.5-pro-preview-03-25 will automatically point to the new version of the model' (05-06); June 26, 2025: redirected to gemini-2.5-pro. Only results first published before 2025-05-06 are mapped here. Kept separate from gemini-2-5-pro-exp-03-25 (the catalogue's existing decision: different API id, no source states they are the same checkpoint), gemini-2-5-pro-preview-05-06 and GA gemini-2-5-pro. Shut down (deprecations: December 2, 2025).
$ how it compares
The LLMs ranked either side of Gemini 2.5 Pro Preview (03-25), out of 828 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 134 | Gemini 3.1 Flash-Lite | 67±1 | api $0.56/M | ||
| 135 |  Qwen3-Coder-Next (80B-A3B) | 67±2 | api $0.60/M +local | ||
| 136 | Gemini 2.5 Pro Preview (03-25) | 67±2 | comparison only | ||
| 137 | Gemini 2.5 Pro Preview (05-06) | 67±1 | comparison only | ||
| 138 |  GigaChat 3.5 Ultra Reasoning (432B-A28B) | 67±1 | local | ||
| more from Google | |||||
| 9 | Gemini 4 Argon (limited preview) | 82±1 | limited preview | ||
| 23 | Gemini 3.8 Flash | 77±1 | api $1.50/M | ||
| 32 | Gemini 3.7 Flash | 76±1 | api $1.50/M | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #152 of 340 * · 3.1% of its score · 1 test
Trails SmolLM3 3B, Jamba 1.5 Large, GPT-4o (2024-05-13) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Galileo Agent Leaderboard v1 (tool selection quality) | 94.1% ✎ | ▃▃▃▃▆▅▆▅█▂▅▂ beats 30/31 | 3.1% |
coding▁▁▁▃▅▇█▇▅▄▃▁ #121 of 524 * · 95% of its score · 2 tests
Beats Grok 4.3, Qwen3.5 Plus, DeepSeek V3.2 here, though they rank higher overall. Trails Llama 3.1 Nemotron Ultra 253B v1, Solar Pro 4, Laguna XS 2.1 (33B-A3B) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) | 72.9% ✎ | ▂·▂▃▅▂▃█▆▄▂▂ beats 42/49 | 81% |
| WebApp1K-Duo-React pass@1 (onekq-ai) | 58.7% ✎ | ▁··▁·▁▁▁▆█▇█ beats 13/45 | 14% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Aider polyglot benchmark runs 81%, WebApp1K models leaderboard (… 14%, Galileo Agent Leaderboard (v1… 3%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Aider polyglot benchmark runs | 1 | 81% |
| WebApp1K models leaderboard (onekq-ai) | 1 | 14% |
| Galileo Agent Leaderboard (v1 and v2 results) | 1 | 3% |
| prior · pull toward the middle | 2% | |
| weighted average | = 66.8 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json