whatmodel.sh / UL2 20B
UL2 20B
↗ share
+ contribute
UL2 20B is an open-weights language model from Google, released May 2022. Its published size is 20B parameters. Licence: apache-2.0.
Catalogue identity notes
Google UL2 20B encoder-decoder pretrained with mixture-of-denoisers (not Flan-UL2, released 2023-02). Recitation paper Table 7 lists 19.46B parameters. arXiv:2205.05131v1 (submitted 10 May 2022) abstract: 'We release Flax-based T5X model checkpoints for the 20B model at https://github.com/google-research/google-research/tree/master/ul2.' Hugging Face google/ul2 (Apache-2.0) first commit 2022-06-16.
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of UL2 20B, out of 877 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 799 |  Granite 4.0 350M | 28±1 | local | ||
| 800 |  OPT-30B | 28±1 | local | ||
| 801 | UL2 20B | 28±3 | comparison only | ||
| 802 |  GPT-NeoX-20B | 28±1 | local | ||
| 803 |  Qwen2 0.5B Instruct | 28±2 | local | ||
| more from Google | |||||
| 11 |  Gemini 4 Argon | 80±1 | api $4.00/M | ||
| 26 | Gemini 3.8 Flash | 76±1 | api $1.50/M | ||
| 34 | Gemini 3.7 Flash | 75±1 | api $1.50/M | ||
$ what it's good at
mathnot ranked on this task yet · 1 test
other tests5 not in a task · 97% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| ARC Challenge lab +1 | 42.9% ✎ | ▄▁▁▂▁▄█▇▃▂▃▂ beats 86/140 | 56% |
| TriviaQA lab | 48.7% ✎ | ▁▁▂▂▄▄▃▆▅█▅▂ beats 45/80 | 20% |
| Natural Questions (closed book) lab | 10.2% ✎ | ▂▁▁▁▂▄▄▃▅█▅▁ beats 27/68 | 14% |
| ARC Easy lab | 61.6% ✎ | ▁▃▂▄▆█▅▃▁··▁ beats 35/100 | 6.5% |
| OpenBookQA lab | 53.6% ✎ | ▅▁▁▁▂▂▃█▃▁▃▁ beats 83/107 | — |
$ where to run it
No verified API or download recorded yet · add one
Official weights and documentation: huggingface.co/google ↗
$ how the score was worked out · Self-Consistency Improves Cha… 62%, Recitation-Augmented Language… 35%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Self-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runs · lab-reported | 2 | 62% |
| Recitation-Augmented Language Models — authors' own runs · lab-reported | 2 | 35% |
| prior · pull toward the middle | 3% | |
| weighted average | = 28.0 |
not counted (3)
| Large Language Models Can Self-Improve — authors' own runs | GSM8K [correct] | 5.4% | fewer than three models have this test yet, or this source ran it for this model alone |
| Large Language Models Can Self-Improve — authors' own runs | ARC Challenge [correct] | 49.9% | the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts |
| Large Language Models Can Self-Improve — authors' own runs | OpenBookQA [correct] | 53.6% | fewer than three models have this test yet, or this source ran it for this model alone |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json