whatmodel.sh / BLOOMZ 1.7B

BLOOMZ 1.7B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

BLOOMZ 1.7B is an open-weights language model from BigScience, released November 2022. Its published size is 1.72B parameters. Licence: bigscience-bloom-rail-1.0.

Catalogue identity notes

BLOOM 1.7B multitask-finetuned on xP3 (arXiv 2211.01786); checkpoint bigscience/bloomz-1b7.

Model information sources

Release date ↗

23.2±2.6comparison only: no current run option verifiedfrom 5 benchmark results across 2 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of BLOOMZ 1.7B, out of 877 with a score, then more from BigScience.

#modelscorewhereRAMreleased
824OPT-6.7B Meta24±1local~16GB2022-05
825T0 3B BigScience23±4comparison only2021-10
826BLOOMZ 1.7B BigScience23±3comparison only2022-11
827Qwen1.5 0.5B Chat Alibaba23±3local~1.7GB2024-01
828Pythia 6.9B EleutherAI23±1local~6.1GB2023-02
more from BigScience
743T0 11B BigScience35±3comparison only2021-10
759BLOOMZ 176B BigScience33±2comparison only2022-11
796BLOOMZ 7.1B BigScience29±2comparison only2022-11

see all LLMs · find the ones that fit your machine

$ what it's good at

coding█···▄·▄·▄··▄ #4 of 6 * · 78% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
HumanEval lab4.4% ✎██··█··█·█·█ beats 2/5medium · avg 58%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 78%sharp · too hard for it
other tests4 not in a task · 19% of its score
testresultvs the others in its rundifficultycountswhy
WinoGrande lab50.7% ✎█··▃▃·▁··▁·▃ beats 6/12medium · avg 68%▇▇▇▇ 11%too hard for it · noisy
COPA lab66.0% ✎▃··█▅▃▃▅·▃▃▃ beats 7/12easy · avg 91%▇▇ 5.9%at its level · noisy
Story Cloze Test lab79.6% ✎█·▂··▃··▃▂▂▃ beats 8/12easy · avg 88%▇ 1.3%blunt · at its level · noisy
IndoMMLU (zero-shot)34.4% ✎▅▃▅··▃···▃▃█ beats 7/10hard · avg 37%▇ 1.2%blunt · too hard for it · noisy

$ where to run it

No verified API or download recorded yet · add one

Official weights and documentation: huggingface.co/bigscience ↗

$ how the score was worked out · Crosslingual Generalization t… 96%, Large Language Models Only Pa… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Crosslingual Generalization through Multitask Finetuning — authors' own runs · lab-reported4CC BY 4.0
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs1CC BY 4.0
prior · pull toward the middle
weighted average= 23.2

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json