whatmodel.sh / BLOOMZ 560M
BLOOMZ 560M
↗ share
+ contribute
BLOOMZ 560M is an open-weights language model from BigScience, released November 2022. Its published size is 0.559B parameters. Licence: bigscience-bloom-rail-1.0.
Catalogue identity notes
BLOOM 560M multitask-finetuned on xP3 (Muennighoff et al., arXiv 2211.01786, 2022-11-03); checkpoint bigscience/bloomz-560m (repo created 2022-10-08).
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of BLOOMZ 560M, out of 877 with a score, then more from BigScience.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 846 |  Flan-T5 Small | 16±3 | local | ||
| 847 |  GPT-3 babbage (API base model) | 16±1 | comparison only | ||
| 848 | BLOOMZ 560M | 15±4 | comparison only | ||
| 849 |  OPT-1.3B | 14±1 | local | ||
| 850 | T5 Base | 14±3 | local | ||
| more from BigScience | |||||
| 743 | T0 11B | 35±3 | comparison only | ||
| 759 | BLOOMZ 176B | 33±2 | comparison only | ||
| 796 | BLOOMZ 7.1B | 29±2 | comparison only | ||
$ what it's good at
coding█···▄·▄·▄··▄ #6 of 6 * · 62% of its score · 1 test
other tests5 not in a task · 31% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| WinoGrande lab | 49.8% ✎ | █··▁▃·▁··▁·▃ beats 0/12 | 12% |
| COPA lab | 51.0% ✎ | ▃··█▅▃▃▅·▃▃▃ beats 0/12 | 11% |
| MC-TACO as yes/no plausibility classification (accuracy) | 59.0% ✎ | █··········▅ beats 0/4 | 3.5% |
| Story Cloze Test lab | 58.5% ✎ | █·▂··▃··▂▂▂▃ beats 6/12 | 2.6% |
| IndoMMLU (zero-shot) | 24.0% ✎ | ▅▃▅··▃···▃▃█ beats 1/10 | 1.8% |
$ where to run it
No verified API or download recorded yet · add one
Official weights and documentation: huggingface.co/bigscience ↗
$ how the score was worked out · Crosslingual Generalization t… 88%, Do Language Models Have a Com… 4%, Large Language Models Only Pa… 2%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Crosslingual Generalization through Multitask Finetuning — authors' own runs · lab-reported | 4 | 88% |
| Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models — authors' own runs | 1 | 4% |
| Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs | 1 | 2% |
| prior · pull toward the middle | 6% | |
| weighted average | = 14.5 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json