whatmodel.sh / Flan-UL2 20B
Flan-UL2 20B
↗ share
+ contribute
Flan-UL2 20B is an open-weights language model from Google, released March 2023. Its published size is 20B parameters. Licence: apache-2.0.
Catalogue identity notes
Instruction-tuned (Flan) UL2 20B released by Google; Hugging Face repo google/flan-ul2 created 2023-03-03, Apache-2.0.
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of Flan-UL2 20B, out of 877 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 487 |  GPT-3.5 Turbo (0125) | 48±1 | api $0.75/M | ||
| 488 |  EuroLLM 9B Instruct | 48±2 | local | ||
| 489 | Flan-UL2 20B | 48±4 | comparison only | ||
| 490 | GPT-3.5 Turbo (1106) | 48±1 | comparison only | ||
| 491 |  Granite 4.0 H Micro 3B | 48±1 | local | ||
| more from Google | |||||
| 11 |  Gemini 4 Argon | 80±1 | api $4.00/M | ||
| 26 | Gemini 3.8 Flash | 76±1 | api $1.50/M | ||
| 34 | Gemini 3.7 Flash | 75±1 | api $1.50/M | ||
$ what it's good at
other tests5 not in a task · 94% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| ZeroSCROLLS Qasper (zero-shot F1) | 56.9% ✎ | ▂▂··▂·▂··▂█▄ beats 10/10 | 32% |
| ZeroSCROLLS QuALITY (zero-shot accuracy) | 75.6% ✎ | ▃····▃·▃▃█▅▅ beats 8/10 | 30% |
| ZeroSCROLLS NarrativeQA (zero-shot F1) | 25.5% ✎ | ▃··▃·█▃·█▃·▃ beats 8/10 | 18% |
| FANToM BeliefQ [Choice], short context (zero-shot) | 23.0% ✎ | █·····▂····▁ beats 6/11 | 10% |
| ZeroSCROLLS MuSiQue (zero-shot F1) | 51.3% ✎ | ▄·█··██·▄▄▄▄ beats 10/10 | 4.9% |
$ where to run it
No verified API or download recorded yet · add one
Official weights and documentation: huggingface.co/google ↗
$ how the score was worked out · ZeroSCROLLS: A Zero-Shot Benc… 85%, FANToM: A Benchmark for Stres… 10%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runs | 4 | 85% |
| FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions — authors' own runs | 1 | 10% |
| prior · pull toward the middle | 6% | |
| weighted average | = 47.8 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json