whatmodel.sh / T0++ (T0pp) 11B

T0++ (T0pp) 11B

+ contribute

T0++ (T0pp) 11B is an open-weights language model from BigScience, released October 2021. Its published size is 11B parameters. Licence: apache-2.0.

Catalogue identity notes

BigScience T0pp ('T0++', 11B) trained on P3 plus GPT-3 and SuperGLUE datasets (Sanh et al., arXiv 2110.08207); checkpoint bigscience/T0pp.

Model information sources

Release date ↗

22.5±2.5comparison only: no current run option verifiedfrom 9 benchmark results across 5 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of T0++ (T0pp) 11B, out of 877 with a score, then more from BigScience.

#	model	score	where	RAM	released
827	Qwen1.5 0.5B Chat Alibaba	23±3	local	~1.7GB	2024-01
828	Pythia 6.9B EleutherAI	23±1	local	~6.1GB	2023-02
829	T0++ (T0pp) 11B BigScience	23±3	comparison only		2021-10
830	Phi-1 (1.3B) Microsoft	22±4	local	~2.4GB	2023-09
831	Dolly v2 12B Databricks	22±1	comparison only		2023-04
	more from BigScience
743	T0 11B BigScience	35±3	comparison only		2021-10
759	BLOOMZ 176B BigScience	33±2	comparison only		2022-11
796	BLOOMZ 7.1B BigScience	29±2	comparison only		2022-11

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests9 not in a task · 97% of its score

test	result	vs the others in its run	difficulty	counts	why
MMLU	30.9% ✎	▁▁▁▁▃▂▂▅██▆▄ beats 14/214	medium · avg 73%	▇▇▇▇▇▇▇▇▇▇▇▇▇ 33%	small run
ZeroSCROLLS QuALITY (zero-shot accuracy)	21.4% ✎	▃····▃·▃▃█▅▅ beats 0/10	easy · avg 81%	▇▇▇▇▇ 13%	typical
Super-NaturalInstructions English test (definition only, zero-shot), ROUGE-L	40.3% ✎	▄·▄······▄▄█ beats 3/5	easy · avg 77%	▇▇▇▇▇ 13%	at its level · small run
ZeroSCROLLS Qasper (zero-shot F1)	25.0% ✎	▂▂··▃·▂··▂█▄ beats 2/10	medium · avg 56%	▇▇▇▇▇ 12%	blunt
BIG-Bench Hard	13.9% ✎	▃▁▁▁▂▄▅▆█▃▃▁ beats 1/155	medium · avg 58%	▇▇▇▇ 11%	sharp · too hard for it · small run
ZeroSCROLLS NarrativeQA (zero-shot F1)	18.7% ✎	▃··▃·█▃·█▃·▃ beats 2/10	hard · avg 29%	▇▇ 5.9%	blunt
LMentry accuracy (zero-shot)	49.9% ✎	▂▆·▄▄█▂·▂··▂ beats 11/14	medium · avg 65%	▇▇ 4.1%	blunt · at its level
CREPE event likelihood change (3-shot, macro F1)	33.7% ✎	██····█····█ beats 0/3	medium · avg 52%	▇▇ 3.8%	blunt

1 more test

test	result	vs the others in its run	difficulty	counts	why
ZeroSCROLLS MuSiQue (zero-shot F1)	35.3% ✎	▄·█··██·▄▄▄▄ beats 5/10	medium · avg 43%	▇ 2.1%	blunt · at its level

$ where to run it

No verified API or download recorded yet · add one

Official weights and documentation: huggingface.co/bigscience ↗

$ how the score was worked out · ZeroSCROLLS: A Zero-Shot Benc… 33%, Model-Generated Pretraining S… 33%, Unnatural Instructions: Tunin… 24% and 2 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding — authors' own runs	4	▇▇▇▇▇▇▇▇▇▇ 33%	CC BY 4.0
Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers — authors' own runs	1	▇▇▇▇▇▇▇▇▇▇ 33%	CC BY 4.0
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor — authors' own runs	2	▇▇▇▇▇▇▇ 24%	CC BY 4.0
LMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runs	1	▇ 4%	CC BY 4.0
Causal Reasoning of Entities and Events in Procedural Texts — authors' own runs	1	▇ 4%	CC BY 4.0
prior · pull toward the middle		▇ 3%
weighted average		= 22.5

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-14 08:02 UTC · sources and licences · json