T0 11B

+ contribute

T0 11B is an open-weights language model from BigScience, released October 2021. Its published size is 11B parameters. Licence: apache-2.0.

Catalogue identity notes

BigScience T0 (11B, T5-LM-adapted XXL multitask-prompted on the P3 mixture; Sanh et al., arXiv 2110.08207, 2021-10-15); checkpoint bigscience/T0. Not T0+ or T0++.

Model information sources

Release date ↗

34.6±2.5comparison only: no current run option verifiedfrom 9 benchmark results across 6 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of T0 11B, out of 877 with a score, then more from BigScience.

#	model	score	where	RAM	released
741	Vicuna 7B v1.3 LMSYS	35±2	local	~5.8GB	2023-06
742	Stable Code 3B Stability AI	35±1	local	~4.8GB	2024-01
743	T0 11B BigScience	35±3	comparison only		2021-10
744	Qwen1.5 4B Chat Alibaba	34±2	local	~4.1GB	2024-02
745	DeepSeek-Coder 1.3B Base DeepSeek	34±1	local	~2.4GB	2023-10
	more from BigScience
759	BLOOMZ 176B BigScience	33±2	comparison only		2022-11
796	BLOOMZ 7.1B BigScience	29±2	comparison only		2022-11
797	BLOOM 176B BigScience	29±1	local	~372GB	2022-07

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests11 not in a task · 97% of its score

test	result	vs the others in its run	difficulty	counts	why
WebQSP closed-book zero-shot (answer-in-generation accuracy)	40.8% ✎	██···██····█ beats 4/4	easy · avg 66%	▇▇▇▇▇▇▇▇▇▇ 25%	at its level
WinoGrande lab +1	60.5% ✎	█··▁▃·▁··▁·▃ beats 12/12	medium · avg 68%	▇▇▇▇▇▇▇▇ 19%	noisy
Super-NaturalInstructions English test (definition + 2 positive examples), ROUGE-L	32.3% ✎	█····█··█··█ beats 0/3	easy · avg 70%	▇▇▇▇▇ 13%	at its level
BIG-Bench Hard, 27 subsets, zero-shot direct (CoT Collection protocol)	33.6% ✎	█····█··█··█ beats 1/3	medium · avg 56%	▇▇▇▇▇ 13%	blunt · at its level
Mintaka (English) closed-book zero-shot (answer-in-generation accuracy)	23.7% ✎	▄····▄···▄·█ beats 2/4	medium · avg 48%	▇▇▇ 8.7%	blunt · at its level
Super-NaturalInstructions English test (definition only, zero-shot), ROUGE-L	33.1% ✎	▄·▄······▄▄█ beats 2/5	medium · avg 63%	▇▇▇ 8.5%	at its level · noisy · small run
COPA lab	93.0% ✎	▃··█▅▃▃▅·▃▃▃ beats 12/12	easy · avg 91%	▇▇ 5.3%	at its level · noisy
LMentry accuracy (zero-shot)	40.9% ✎	▂▆·▄▄█▂·▂··▂ beats 6/14	medium · avg 65%	▇▇ 4.3%	blunt · at its level

3 more tests

test	result	vs the others in its run	difficulty	counts	why
Story Cloze Test lab	94.7% ✎	█·▂··▃··▂▂▂▃ beats 11/12	easy · avg 88%	▇ 1.2%	blunt · at its level · noisy
HellaSwag	63.7% ✎	▁▁▁▂▂▂▄▆█▇▄▂ beats 51/135	very easy · avg 91%	—	not counted yet: no other model in the same run to compare it with
PIQA	84.9% ✎	▁▁▁▂▂▃▅█▅▃▃▁ beats 120/123	easy · avg 87%	—	not counted yet: no other model in the same run to compare it with

$ where to run it

No verified API or download recorded yet · add one

Official weights and documentation: huggingface.co/bigscience ↗

$ how the score was worked out · Knowledge-Augmented Language… 33%, Crosslingual Generalization t… 25%, Super-NaturalInstructions: Ge… 13% and 3 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering — authors' own runs	2	▇▇▇▇▇▇▇▇▇▇ 33%	CC BY 4.0
Crosslingual Generalization through Multitask Finetuning — authors' own runs · lab-reported	3	▇▇▇▇▇▇▇▇ 25%	CC BY 4.0
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks — authors' own runs	1	▇▇▇▇ 13%	CC BY 4.0
The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning — authors' own runs	1	▇▇▇▇ 13%	CC BY 4.0
Self-Instruct: Aligning Language Models with Self-Generated Instructions — authors' own runs	1	▇▇▇ 8%	CC BY 4.0
LMentry: A Language Model Benchmark of Elementary Language Tasks — authors' own runs	1	▇ 4%	CC BY 4.0
prior · pull toward the middle		▇ 3%
weighted average		= 34.6

not counted (3)

NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation — authors' own runs	HellaSwag [correct]	63.7%	fewer than three models have this test yet, or this source ran it for this model alone
NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation — authors' own runs	WinoGrande [correct]	58.9%	the only model in this source's run of the test, so there is nothing to compare it with there; another run of the test counts
NovaCOMET: Open Commonsense Foundation Models with Symbolic Knowledge Distillation — authors' own runs	PIQA [correct]	84.9%	fewer than three models have this test yet, or this source ran it for this model alone

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-14 08:02 UTC · sources and licences · json