whatmodel.sh / Gemini 1.5 Flash (001)
Gemini 1.5 Flash (001)
↗ share
+ contribute
Gemini 1.5 Flash (001) is a closed-weights language model from Google, released May 2024. Licence: proprietary.
Catalogue identity notes
Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-05-23). Stable API version gemini-1.5-flash-001 (May 2024); distinct from -002. Gemini API changelog, September 29, 2025: Gemini 1.5 models shut down.
$ how it compares
The LLMs ranked either side of Gemini 1.5 Flash (001), out of 828 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 405 |  CodeQwen1.5 7B Chat | 50±1 | local | ||
| 406 |  Jamba 1.5 Large | 50±1 | local | ||
| 407 | Gemini 1.5 Flash (001) | 50±1 | comparison only | ||
| 408 |  Llama-3-8B-ProLong-512k-Instruct | 50±5 | local | ||
| 409 |  EXAONE Deep 2.4B | 50±2 | local | ||
| more from Google | |||||
| 9 | Gemini 4 Argon (limited preview) | 82±1 | limited preview | ||
| 23 | Gemini 3.8 Flash | 77±1 | api $1.50/M | ||
| 32 | Gemini 3.7 Flash | 76±1 | api $1.50/M | ||
$ what it's good at
coding▁▁▁▃▅▇█▇▅▄▃▁ #284 of 524 * · 15% of its score · 2 tests
Beats Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B), GLM-4.7-Flash here, though they rank higher overall. Trails QwQ-32B-Preview, InternLM2.5 20B Chat, Mistral Small Instruct 22B (2409) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| CRUXEval (ZeroEval, 800 problems) | 63.7% ✎ | ▁▂▆▃▆█▆▃▂·▁▁ beats 22/47 | 13% |
| WebApp1K-React pass@1 (onekq-ai) | 57.0% ✎ | ▃▁▁█▁▄▅▃▄▇▁▄ beats 13/31 | 2.3% |
chat & writing▁▁▁▃▅▄▆█▇▇▇▃ #279 of 463 * · 4.7% of its score · 1 test
Beats Mistral Small 4 (119B-A6.5B), Ministral 3 14B Reasoning 2512, Qwen2.5 Coder 32B Instruct here, though they rank higher overall. Trails Flan-T5 XXL, Granite 4.0 H 1B, LFM2.5 1.2B Instruct here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| LMArena (text) | 1286.5 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 115/379 | 4.7% |
math▁▂▃▅▇▅▅█▄▁▁▁ #346 of 462 · 24% of its score · 4 tests
Beats GPT-4 (0613), Athene-70B here, though they rank higher overall. Trails Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B, CodeGemma 7B (v1.0) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MATH level 5 | 25.1% ✎ | ▂▂▅▆▄█▆▃▅▂▃▄ beats 23/92 | 11% |
| MATH Level 5 (ZeroEval, 721 problems) | 34.8% ✎ | ▁▁▁▆█▃▇▇▄··▂ beats 17/38 | 6.5% |
| GSM8K (ZeroEval, zero-shot) | 91.4% ✎ | ▁▄·▁▄█▅▅▅███ beats 23/43 | 5.4% |
| OTIS mock AIME 2024–2025 | 3.9% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 26/193 | 1.6% |
vision & documents▁▁▃▃▃▃▅▄▇▇█▃ #149 of 182 · 12% of its score · 4 tests
Beats DeepSeek V4 Flash Vision (Exp), Gemma 3 12B here, though they rank higher overall. Trails Qwen3.5 2B, Qwen2.5-VL 7B Instruct here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Arena vision (style controlled) | 1059.5 ✎ | ▁▁▂▂▂▃▄▅▅▆██ beats 22/145 | 3.2% |
| CharXiv (reasoning questions) | 33.9% ✎ | ▂▂▂█▃▂·▂▂··▂ beats 8/13 | 3.1% |
| MMMU val (VHELM, zero-shot exact match) | 56.6% ✎ | ▄▄··▄▄█··▄▄█ beats 5/9 | 3.1% |
| MathVista testmini (VHELM, zero-shot exact match) | 51.2% ✎ | ▃·▅···▃▃▃·▃█ beats 5/9 | 2.6% |
other tests15 not in a task · 44% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MMLU-Redux (ZeroEval, 2,778 questions) | 77.4% ✎ | ▁·▁▁▄▄▄▅█▄▂▁ beats 24/53 | 21% |
| ZebraLogic (ZeroEval, 1,000 puzzles) | 19.4% ✎ | ▁▁▃▆██▁▁·▁▁▁ beats 29/57 | 7.2% |
| GPQA Diamond | 40.4% ✎ | ▁▁▁▁▂▄▆▄▅▇█▅ beats 38/208 | 5.6% |
| AHELM Speech Robust Bench (LibriSpeech-clean with noise) transcription WER | 49.8% ✎ | ▂·█··▆▄·▂▂·▄ beats 1/13 | 4.4% |
| AHELM CoRe-Bench conversational audio reasoning (pseudo-exact match) | 72.2% ✎ | ▂·▄··▂▂·▄▃█▄ beats 7/13 | 1.6% |
| Open PT-LLM Leaderboard average (9 Portuguese tasks) | 79.3% ✎ | ▃▁▁▁▁▁▁▁▂█▅▁ beats 254/276 | 1.0% |
| AHELM LibriSpeech transcription by audio LLMs (WER) | 34.2% ✎ | ▃···▃·█▃▅▃▅█ beats 0/13 | 0.9% |
| AHELM MUStARD audio sarcasm detection (exact match) | 55.5% ✎ | ▂·······▆█▆▆ beats 1/13 | 0.6% |
7 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM VoxCeleb2 same-speaker detection (exact match) | 52.2% ✎ | ▂·▂·▃█▂▄▄··▄ beats 2/13 | 0.6% |
| AHELM AIR-Bench Foundation music subsets (exact match) | 48.3% ✎ | ▃█▃▅▃·▃·▃▅▃▃ beats 0/13 | 0.4% |
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 93.4% ✎ | ▁▁▂▃▄▄█▆▃▂▂▁ beats 46/140 | 0.3% |
| AHELM MELD audio emotion classification (pseudo-exact match) | 47.1% ✎ | ▃··▃▃▃·▃▅▅█▅ beats 9/13 | 0.2% |
| AHELM FLEURS 7-language transcription by audio LLMs (WER) | 72.3% ✎ | ▄▂···▂▆▂█▂·▂ beats 4/13 | 0.1% |
| AHELM Multilingual LibriSpeech 7-language transcription by audio LLMs (WER) | 8.8% ✎ | ▂▂▂··▂▂·▂▃█▆ beats 6/13 | 0.0% |
| AHELM VocalSound human-sound recognition (prefix exact match) | 46.3% ✎ | ██▄▄·█·█·█▄▄ beats 1/13 | 0.0% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · WildEval ZeroEval published r… 53%, Epoch AI — internal evaluatio… 18%, AHELM: A Holistic Evaluation… 9% and 6 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 WildEval ZeroEval published results | 5 | 53% |
 Epoch AI — internal evaluations | 3 | 18% |
 AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2) | 10 | 9% |
 LMArena leaderboard dataset | 2 | 8% |
| VHELM: A Holistic Evaluation of Vision Language Models (Stanford CRFM), arXiv 2410.07112v2 | 2 | 6% |
 CharXiv leaderboard (princeton-nlp/CharXiv README) | 1 | 3% |
| WebApp1K models leaderboard (onekq-ai) | 1 | 2% |
| Open Portuguese LLM Leaderboard (archived) | 1 | 1% |
| Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository) | 1 | 0% |
| prior · pull toward the middle | 0% | |
| weighted average | = 50.2 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json