whatmodel.sh / Gemini 1.5 Flash (002)
Gemini 1.5 Flash (002)
↗ share
+ contribute
Gemini 1.5 Flash (002) is a closed-weights language model from Google, released September 2024. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-09-24). Stable API version gemini-1.5-flash-002 (September 2024); distinct from -001. Gemini API changelog, September 29, 2025: Gemini 1.5 models shut down.
$ how it compares
The LLMs ranked either side of Gemini 1.5 Flash (002), out of 828 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 361 |  Olmo 3 7B Think | 53±1 | local | ||
| 362 |  GPT-4.1 nano | 53±1 | api $0.17/M | ||
| 363 | Gemini 1.5 Flash (002) | 53±1 | comparison only | ||
| 364 |  Gemma 3 27B | 53±1 | api $0.10/M +local | ||
| 365 |  Phi-4 14B | 53±1 | api $0.087/M +local | ||
| more from Google | |||||
| 9 | Gemini 4 Argon (limited preview) | 82±1 | limited preview | ||
| 23 | Gemini 3.8 Flash | 77±1 | api $1.50/M | ||
| 32 | Gemini 3.7 Flash | 76±1 | api $1.50/M | ||
$ what it's good at
agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #178 of 340 * · 0.7% of its score · 1 test
Beats o1 (2024-12-17), o3-mini, DeepSeek R1 0528 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Galileo Agent Leaderboard v1 (tool selection quality) | 89.5% ✎ | ▃▃▃▃▆▅▆▅█▂▅▂ beats 21/31 | 0.7% |
coding▁▁▁▃▅▇█▇▅▄▃▁ #291 of 524 · 57% of its score · 5 tests
Beats Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B), GLM-4.7-Flash here, though they rank higher overall. Trails QwQ-32B-Preview, InternLM2.5 20B Chat, Mistral Small Instruct 22B (2409) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| HumanEval | 82.3% ✎ | ▁·▁▁▃▄▆▅█▃▁▁ beats 87/99 | 19% |
| HumanEval+ | 75.6% ✎ | ▁·▁▁▁▃▄▄▄█▃▁ beats 86/99 | 17% |
| LiveCodeBench official (Jan 1–Nov 1, 2024) | 34.4% ✎ | ▁▁·▁▂▄▆█▅▂·▂ beats 18/31 | 9.3% |
| Aider code editing (133 Exercism Python exercises, two attempts) | 51.1% ✎ | ▁·▁▂▄▁▆▄█▆▂▂ beats 16/62 | 7.0% |
| MBPP+ | 67.5% ✎ | ▁···▁▁▂▅▇█▆▂ beats 56/75 | 3.9% |
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #249 of 463 * · 3.8% of its score · 1 test
Beats Granite 4.2 8B, Mistral Small 4 (119B-A6.5B), Gemma 2 9B IT SimPO here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, Granite 4.0 H Micro 3B, Gemma 3n E4B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| LMArena (text) | 1309.2 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 139/379 | 3.8% |
math▁▂▃▅▇▅▅█▄▁▁▁ #256 of 462 * · 11% of its score · 2 tests
Beats GPT-4o (2024-08-06), Command A (03-2025), GPT-4o (2024-11-20) here, though they rank higher overall. Trails Olmo 3 7B Instruct, DeepSeek R1 Distill Llama 8B, Phi-4-mini-reasoning 3.8B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MATH level 5 | 61.9% ✎ | ▂▂▅▆▄█▆▃▅▂▃▄ beats 51/92 | 9.4% |
| OTIS mock AIME 2024–2025 | 16.3% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 55/193 | 1.9% |
vision & documents▁▁▃▃▂▃▅▄▇▇█▃ #118 of 182 * · 2.6% of its score · 1 test
Beats DeepSeek V4 Flash Vision (Exp), Magistral Small 1.2, Qwen3-VL 30B-A3B Instruct here, though they rank higher overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena vision (style controlled) | 1141.4 ✎ | ▁▁▂▂▂▂▄▅▅▆██ beats 43/145 | 2.6% |
other tests16 not in a task · 24% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MMLU-Pro | 64.0% ✎ | ▃▂▅█▃·█▃▅··▂ beats 16/23 | 6.4% |
| GPQA Diamond | 47.3% ✎ | ▁▁▁▁▂▄▆▄▅▇█▅ beats 58/208 | 5.5% |
| AHELM Speech Robust Bench (LibriSpeech-clean with noise) transcription WER | 21.4% ✎ | ▂·█··▆▄·▂▂·▄ beats 5/13 | 3.6% |
| LiveBench 2024-08-31 | 49.6% ✎ | ▄█·▄·▄▄·▄··▄ beats 3/7 | 2.3% |
| EvalPerf correctness (118 tasks, 20240328) | 77.6% ✎ | ▂·▂▂▂▆▅▆▆▆▃█ beats 14/29 | 2.1% |
| AHELM CoRe-Bench conversational audio reasoning (pseudo-exact match) | 77.6% ✎ | ▂·▄··▂▂·▄▂█▄ beats 11/13 | 1.2% |
| Open PT-LLM Leaderboard average (9 Portuguese tasks) | 79.9% ✎ | ▃▁▁▁▁▁▁▁▂█▅▁ beats 259/276 | 0.8% |
| AHELM LibriSpeech transcription by audio LLMs (WER) | 6.2% ✎ | ▃···▃·█▃▅▃▅█ beats 8/13 | 0.7% |
8 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM VoxCeleb2 same-speaker detection (exact match) | 54.2% ✎ | ▂·▂·▂█▃▄▄··▄ beats 7/13 | 0.6% |
| AHELM MUStARD audio sarcasm detection (exact match) | 55.8% ✎ | ▂·······▆█▆▆ beats 2/13 | 0.5% |
| AHELM AIR-Bench Foundation music subsets (exact match) | 50.8% ✎ | ▃█▃▅▃·▃·▃▅▃▃ beats 1/13 | 0.3% |
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 96.6% ✎ | ▁▁▂▃▄▄█▆▃▂▂▁ beats 97/140 | 0.2% |
| AHELM MELD audio emotion classification (pseudo-exact match) | 42.5% ✎ | ▃··▃▃▃·▃▅▅█▅ beats 5/13 | 0.2% |
| AHELM FLEURS 7-language transcription by audio LLMs (WER) | 46.1% ✎ | ▄▂···▂▆▂█▂·▂ beats 9/13 | 0.1% |
| AHELM Multilingual LibriSpeech 7-language transcription by audio LLMs (WER) | 7.1% ✎ | ▂▂▂··▂▂·▂▂█▆ beats 9/13 | 0.0% |
| AHELM VocalSound human-sound recognition (prefix exact match) | 41.8% ✎ | ██▄▄·█·█·█▄▄ beats 0/13 | 0.0% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · EvalPlus / EvalPerf published… 43%, Epoch AI — internal evaluatio… 17%, LiveCodeBench official websit… 9% and 8 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json