whatmodel.sh / Gemini 2.5 Flash Preview (04-17)
Gemini 2.5 Flash Preview (04-17)
↗ share
+ contribute
Gemini 2.5 Flash Preview (04-17) is a closed-weights language model from Google, released April 2025. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2025-04-17). Preview API model gemini-2.5-flash-preview-04-17; distinct from GA gemini-2-5-flash. Gemini API changelog announced its deprecation for July 15, 2025; not listed as available.
$ how it compares
The LLMs ranked either side of Gemini 2.5 Flash Preview (04-17), out of 828 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 192 |  Granite 4.2 30B | 63±1 | api $0.28/M +local | ||
| 193 |  o1-preview | 63±1 | comparison only | ||
| 194 | Gemini 2.5 Flash Preview (04-17) | 63±1 | comparison only | ||
| 195 |  GLM-4.5-Air | 63±1 | api $0.42/M +local | ||
| 196 |  Qwen3.5 9B | 63±1 | api $0.11/M +local | ||
| more from Google | |||||
| 9 | Gemini 4 Argon (limited preview) | 82±1 | limited preview | ||
| 23 | Gemini 3.8 Flash | 77±1 | api $1.50/M | ||
| 32 | Gemini 3.7 Flash | 76±1 | api $1.50/M | ||
$ what it's good at
coding▁▁▁▃▅▇█▇▅▄▃▁ #166 of 524 · 77% of its score · 3 tests
Beats Gemini 2.5 Pro, Gemini 3.1 Flash-Lite, Qwen3-Coder-Next (80B-A3B) here, though they rank higher overall. Trails Phi-4-reasoning 14B, DeepSeek V3 0324, Devstral Small 2 24B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) | 47.1% ✎ | ▂·▂▃▅▂▃█▆▄▂▂ beats 24/49 | 54% |
| LiveCodeBench official (Aug 1, 2024–May 1, 2025) | 60.6% ✎ | ▃▃▅▃··▃▅█▅▅▃ beats 7/15 | 14% |
| WebApp1K-Duo-React pass@1 (onekq-ai) | 57.0% ✎ | ▁··▁·▁▁▁▆█▇█ beats 8/45 | 9.1% |
math▁▂▃▅▇▅▅█▄▁▁▁ #142 of 462 * · 18% of its score · 1 test
Beats Claude Sonnet 4.6, MiniMax-M3, MiMo-V2.5 (310B-A15B) here, though they rank higher overall. Trails Olmo 3.1 32B Think, Nemotron-Cascade-8B, Qwen3-VL 30B-A3B Thinking here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| OTIS mock AIME 2024–2025 | 73.1% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 106/193 | 18% |
vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #105 of 182 * · 3.1% of its score · 1 test
Beats DeepSeek V4 Flash Vision (Exp) here, though they rank higher overall. Trails Mistral Small 4 (119B-A6.5B), Qwen3-VL 32B Instruct, GPT-4.1 mini here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MSEarth-Bench Earth-science figure MCQ (Overall accuracy) | 57.2% ✎ | █·██····██·█ beats 3/5 | 3.1% |
other tests1 not in a task · 0.6% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 98.7% ✎ | ▁▁▂▃▄▄█▆▃▂▃▁ beats 132/140 | 0.6% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Aider polyglot benchmark runs 54%, Epoch AI — internal evaluatio… 18%, LiveCodeBench official websit… 14% and 3 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Aider polyglot benchmark runs | 1 | 54% |
 Epoch AI — internal evaluations | 1 | 18% |
| LiveCodeBench official website results (2025-08-01) | 1 | 14% |
| WebApp1K models leaderboard (onekq-ai) | 1 | 9% |
 MSEarth: A Multimodal Benchmark for Earth Science Phenomenon Discovery with MLLMs (arXiv:2505.20740v3) | 1 | 3% |
 Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository) | 1 | 1% |
| prior · pull toward the middle | 1% | |
| weighted average | = 62.8 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json