whatmodel.sh / BLOOMZ 3B

BLOOMZ 3B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

BLOOMZ 3B is an open-weights language model from BigScience, released November 2022. Its published size is 3B parameters. Licence: bigscience-bloom-rail-1.0.

Catalogue identity notes

BLOOM 3B multitask-finetuned on xP3 (arXiv 2211.01786); checkpoint bigscience/bloomz-3b.

Model information sources

Release date ↗

26.6±2.3comparison only: no current run option verifiedfrom 5 benchmark results across 2 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of BLOOMZ 3B, out of 877 with a score, then more from BigScience.

#modelscorewhereRAMreleased
806RWKV-4-Raven 14B RWKV (BlinkDL)28±2comparison only2023-04
807LaMDA 68B (pre-trained) Google27±1never released2022-01
808BLOOMZ 3B BigScience27±2comparison only2022-11
809Qwen1.5 1.8B Chat Alibaba26±2local~2.7GB2024-01
810Tk-Instruct 11B (def) AllenAI / UW26±4comparison only2022-05
more from BigScience
743T0 11B BigScience35±3comparison only2021-10
759BLOOMZ 176B BigScience33±2comparison only2022-11
796BLOOMZ 7.1B BigScience29±2comparison only2022-11

see all LLMs · find the ones that fit your machine

$ what it's good at

coding█···▄·▄·▄··▄ #3 of 6 * · 81% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
HumanEval lab6.3% ✎██··█··█·█·█ beats 3/5medium · avg 58%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 81%sharp · too hard for it
other tests4 not in a task · 17% of its score
testresultvs the others in its rundifficultycountswhy
WinoGrande lab51.8% ✎█··▁▃·▃··▁·▃ beats 9/12medium · avg 68%▇▇▇▇ 10%noisy
COPA lab73.0% ✎▃··█▅▃▃▅·▃▃▃ beats 9/12easy · avg 91%▇▇ 4.5%at its level · noisy
IndoMMLU (zero-shot)36.4% ✎▅▃▅··▃···▃▃█ beats 9/10hard · avg 37%▇ 1.1%blunt · too hard for it · noisy
Story Cloze Test lab85.2% ✎█·▂··▃··▂▃▂▃ beats 9/12easy · avg 88%▇ 1.0%blunt · at its level · noisy

$ where to run it

No verified API or download recorded yet · add one

Official weights and documentation: huggingface.co/bigscience ↗

$ how the score was worked out · Crosslingual Generalization t… 97%, Large Language Models Only Pa… 1%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Crosslingual Generalization through Multitask Finetuning — authors' own runs · lab-reported4CC BY 4.0
Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLU — authors' own runs1CC BY 4.0
prior · pull toward the middle
weighted average= 26.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json