whatmodel.sh / T5 Large

T5 Large

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

T5 Large is an open-weights language model from Google, released October 2019. Its published size is 0.738B parameters and a 1k-token context window. Licence: apache-2.0.

Catalogue identity notes

Original T5 (Raffel et al. 2019) encoder-decoder checkpoint (multi-task pre-trained mixture), not T5 v1.1 and not the LM-adapted T5.

48.1±8.6#380 of 719 LLMsfrom 3 benchmark results across 1 source

$ how it compares

The LLMs ranked either side of T5 Large, out of 828 with a score, then more from Google.

#modelscorewhereRAMreleased
453Reka Core (2024-05-01) Reka AI48±1comparison only2024-05
454Gemini 1.5 Flash-8B (001) Google48±2comparison only2024-10
455T5 Large Google48±9local~4.6GB2019-10
456WaveCoder-Ultra-6.7B Microsoft48±1local~5.8GB2024-04
457Phi-3.5-MoE-instruct Microsoft48±1local~26GB2024-08
more from Google
9Gemini 4 Argon (limited preview) Google82±1limited preview2026-09
23Gemini 3.8 Flash Google77±1api $1.50/M2026-09
32Gemini 3.7 Flash Google76±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests4 not in a task · 67% of its score
testresultvs the others in its rundifficultycountswhy
COPEN concept identification in context (zero-shot, full prompt)42.6% ✎▃···▃▃·▃·▃·█ beats 7/7hard · avg 44%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 63%blunt
COPEN conceptual similarity judgment (zero-shot, full prompt)14.0% ✎▄···█···██·▄ beats 7/7very hard · avg 11%▇ 2.0%blunt · too hard for it
COPEN conceptual property judgment (zero-shot, full prompt)49.8% ✎█·······▁··▁ beats 0/7very hard · avg 51%▇ 1.5%blunt · too hard for it · noisy
Open PT-LLM Leaderboard average (9 Portuguese tasks)14.8% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 0/276medium · avg 73%—failed run: Failed run: average 14.76 vs the leaderboard's random baseline 32.64; the original T5 encoder-decoder checkpoint does not follow the few-shot generative answer format (every task near or below chance), so this is not a measurement of ability.

$ where to run it

Other ways to run it

  1. Run it as published · needs ~2.95 GB · from huggingface.co/google-t5
    uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text2text-generation', model='google-t5/t5-large')('translate English to German: The house is wonderful.', max_new_tokens=40)[0]['generated_text'])"

Weights and documentation: huggingface.co/google-t5 ↗

$ how the score was worked out · COPEN — EMNLP 2022 original z… 67%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
COPEN — EMNLP 2022 original zero-shot evaluations3▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 67%CC BY 4.0
prior · pull toward the middle▇▇▇▇▇▇▇▇▇▇ 33%
weighted average= 48.1

not counted (1)

Open Portuguese LLM Leaderboard (archived)Open PT-LLM Leaderboard average (9 Portuguese tasks) [correct]14.8%failed run: Failed run: average 14.76 vs the leaderboard's random baseline 32.64; the original T5 encoder-decoder checkpoint does not follow the few-shot generative answer format (every task near or below chance), so this is not a measurement of ability.

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json