whatmodel.sh / Gemini 1.5 Flash-8B Experimental (0827)

Gemini 1.5 Flash-8B Experimental (0827)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Gemini 1.5 Flash-8B Experimental (0827) is a closed-weights language model from Google, released August 2024. Licence: proprietary.

Catalogue identity notes

Comparison-only: retired or no longer listed API model, kept so its independent runs can be scored. Gemini API changelog, August 27, 2024: experimental gemini-1.5-flash-8b-exp-0827 released; replaced on September 24, 2024 by gemini-1.5-flash-8b-exp-0924. Distinct from the stable gemini-1.5-flash-8b-001 (catalogued).

46.9±1.4comparison only: no current run option verifiedfrom 2 benchmark results across 2 sources* thinly tested, treat the rank as provisional

$ how it compares

The LLMs ranked either side of Gemini 1.5 Flash-8B Experimental (0827), out of 1132 with a score, then more from Google.

#modelscorewhereRAMreleased
666DeepSeek R1 Distill Llama 8B DeepSeek47±1local~6.2GB2025-01
667EXAONE 4.0 1.2B LG AI Research47±2local~2.2GB2025-07
668Gemini 1.5 Flash-8B Experimental (0827) Google47±1comparison only2024-08
669InternVL2-26B Shanghai AI Laboratory (OpenGVLab)47±2local~55GB2024-07
670speechless-coder-ds-6.7b uukuguy47±1local~5.8GB2023-12
more from Google
9Gemini 4 Argon (limited preview) Google82±1limited preview2026-09
23Gemini 3.8 Flash Google77±1api $1.50/M2026-09
32Gemini 3.7 Flash Google76±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▅▇█▇▅▄▂▁ #408 of 549 * · 32% of its score · 1 test

Beats GLM-4.7-Flash, LFM2.5 2.6B, LFM2.5 8B-A1B here, though they rank higher overall. Trails Phi-3-mini-128k-instruct (Apr 2024), DeepSeek LLM 67B Chat, WizardLM 70B V1.0 here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Aider code editing (133 Exercism Python exercises, two attempts)38.3% ✎▁·▁▂▄▁▆▄█▆▂▂ beats 8/62medium · avg 54%▇▇▇▇▇▇▇▇▇▇▇▇▇ 32%at its level
other tests1 not in a task · 67% of its score
testresultvs the others in its rundifficultycountswhy
LiveBench 2024-07-2637.0% ✎▁▁▁▂▂▃▂█▆▃▅▃ beats 40/61medium · avg 41%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 67%at its level

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · LiveBench model judgments (HF… 67%, Aider code editing benchmark… 32%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
LiveBench model judgments (HF livebench/model_judgment, 2024-10-22 revision) with the LiveBench question datasets1▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 67%Apache-2.0 (stated in the LiveBench datasheet)
Aider code editing benchmark runs (old edit leaderboard)1▇▇▇▇▇▇▇▇▇▇ 32%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 46.9

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json