whatmodel.sh / Gemini 2.5 Flash Preview (05-20)
Gemini 2.5 Flash Preview (05-20)
↗ share
+ contribute
Gemini 2.5 Flash Preview (05-20) is a closed-weights language model from Google, released May 2025. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2025-05-20). Preview API model gemini-2.5-flash-preview-05-20; distinct from GA gemini-2-5-flash. Gemini API deprecations page: shutdown date November 18, 2025.
$ how it compares
The LLMs ranked either side of Gemini 2.5 Flash Preview (05-20), out of 828 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 205 |  GLM-4.6V | 62±2 | api $0.45/M +local | ||
| 206 |  Devstral 2 (2512) | 62±2 | local | ||
| 207 | Gemini 2.5 Flash Preview (05-20) | 62±1 | comparison only | ||
| 208 |  A.X K1 (519B-A33B) | 62±1 | local | ||
| 209 |  gpt-oss-120b | 62±1 | api $0.26/M +local | ||
| more from Google | |||||
| 9 | Gemini 4 Argon (limited preview) | 82±1 | limited preview | ||
| 23 | Gemini 3.8 Flash | 77±1 | api $1.50/M | ||
| 32 | Gemini 3.7 Flash | 76±1 | api $1.50/M | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #228 of 340 * · 11% of its score · 1 test
Beats o1 (2024-12-17), o3-mini, DeepSeek R1 (Jan '25) here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Mellum2 12B-A2.5B Instruct here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| τ²-bench Telecom | 16.9% ✎ | ██··█·██··██ beats 0/6 | 11% |
coding▁▁▁▃▅▇█▇▅▄▃▁ #147 of 524 · 49% of its score · 3 tests
Beats Qwen3.5 Plus, Gemini 2.5 Pro, Gemini 3.1 Flash-Lite here, though they rank higher overall. Trails Devstral Small 2 24B, Llama 3.1 Nemotron Ultra 253B v1 here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) | 55.1% ✎ | ▂·▂▃▅▂▃█▆▄▂▂ beats 30/49 | 34% |
| LiveCodeBench official (Aug 1, 2024–May 1, 2025) | 61.9% ✎ | ▃▃▅▃··▃▅█▅▅▃ beats 8/15 | 9.1% |
| WebApp1K-Duo-React pass@1 (onekq-ai) | 63.4% ✎ | ▁··▁·▁▁▁▆█▇█ beats 18/45 | 5.9% |
math▁▂▃▅▇▅▅█▄▁▁▁ #151 of 462 * · 12% of its score · 1 test
Beats Claude Sonnet 4.6, MiMo-V2.5 (310B-A15B), MiniMax-M2.7 here, though they rank higher overall. Trails Olmo 3 32B Think, Olmo 3.1 32B Think, gpt-oss-20b here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| OTIS mock AIME 2024–2025 | 70.8% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 102/193 | 12% |
other tests11 not in a task · 28% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM Speech Robust Bench (LibriSpeech-clean with noise) transcription WER | 7.7% ✎ | ▂·█··▆▄·▂▃·▄ beats 11/13 | 11% |
| HindiGen-v1 3C3H score (Inception, Claude 3.5 Sonnet judge) | 73.7% ✎ | ▁▁▁▁·▂·█▆█▇▁ beats 24/31 | 4.9% |
| AHELM VoxCeleb2 same-speaker detection (exact match) | 75.9% ✎ | ▂·▂·▂█▂▄▄··▄ beats 13/13 | 3.2% |
| AHELM CoRe-Bench conversational audio reasoning (pseudo-exact match) | 64.4% ✎ | ▂·▄··▂▂·▄▂█▄ beats 5/13 | 2.9% |
| AHELM LibriSpeech transcription by audio LLMs (WER) | 7.7% ✎ | ▃···▃·█▃▅▃▅█ beats 6/13 | 2.2% |
| AHELM MUStARD audio sarcasm detection (exact match) | 57.4% ✎ | ▂·······▆█▆▆ beats 4/13 | 2.0% |
| AHELM AIR-Bench Foundation music subsets (exact match) | 57.9% ✎ | ▃█▃▅▃·▃·▃▅▃▃ beats 7/13 | 0.9% |
| AHELM MELD audio emotion classification (pseudo-exact match) | 34.0% ✎ | ▃··▃▃▃·▃▅▅█▅ beats 2/13 | 0.6% |
3 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM FLEURS 7-language transcription by audio LLMs (WER) | 273.2% ✎ | ▄▂···▂▆▂█▂·▂ beats 0/13 | 0.3% |
| AHELM Multilingual LibriSpeech 7-language transcription by audio LLMs (WER) | 60.3% ✎ | ▃▂▂··▂▂·▂▂█▆ beats 0/13 | 0.0% |
| AHELM VocalSound human-sound recognition (prefix exact match) | 62.6% ✎ | ██▄▄·█·█·█▄▄ beats 5/13 | 0.0% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Aider polyglot benchmark runs 34%, AHELM: A Holistic Evaluation… 23%, Epoch AI — internal evaluatio… 12% and 4 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Aider polyglot benchmark runs | 1 | 34% |
 AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2) | 10 | 23% |
 Epoch AI — internal evaluations | 1 | 12% |
| Moonshot AI Kimi-K2-Instruct model card (instruction-model evaluation table) | 1 | 11% |
| LiveCodeBench official website results (2025-08-01) | 1 | 9% |
| WebApp1K models leaderboard (onekq-ai) | 1 | 6% |
| Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval) | 1 | 5% |
| prior · pull toward the middle | 1% | |
| weighted average | = 62.1 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json