whatmodel.sh / T5 Base

T5 Base

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

T5 Base is an open-weights language model from Google, released October 2019. Its published size is 0.223B parameters and a 1k-token context window. Licence: apache-2.0.

Catalogue identity notes

Original T5 (Raffel et al. 2019) encoder-decoder checkpoint (multi-task pre-trained mixture), not T5 v1.1 and not the LM-adapted T5.

45.9±8.7#423 of 719 LLMsfrom 3 benchmark results across 1 source

$ how it compares

The LLMs ranked either side of T5 Base, out of 828 with a score, then more from Google.

#modelscorewhereRAMreleased
502GPT-3.5 Turbo (0301) OpenAI46±1comparison only2023-03
503Nova Micro Amazon46±2api $0.061/M2024-12
504T5 Base Google46±9local~2.4GB2019-10
505OpenChat 3.5 0106 OpenChat46±1local~6.1GB2024-01
506Qwen2-VL 7B Instruct Alibaba46±1local~6.4GB2024-08
more from Google
9Gemini 4 Argon (limited preview) Google82±1limited preview2026-09
23Gemini 3.8 Flash Google77±1api $1.50/M2026-09
32Gemini 3.7 Flash Google76±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests4 not in a task · 66% of its score
testresultvs the others in its rundifficultycountswhy
COPEN concept identification in context (zero-shot, full prompt)42.3% ✎▃···▃▃·▃·▃·█ beats 6/7hard · avg 44%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 62%blunt
COPEN conceptual similarity judgment (zero-shot, full prompt)10.3% ✎▄···█···██·▄ beats 3/7very hard · avg 11%▇ 2.1%blunt · too hard for it
COPEN conceptual property judgment (zero-shot, full prompt)49.5% ✎█·······▁··▁ beats 1/7very hard · avg 51%▇ 1.5%blunt · too hard for it · noisy
Open PT-LLM Leaderboard average (9 Portuguese tasks)13.7% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 0/276medium · avg 73%—failed run: Failed run: average 13.68 vs the leaderboard's random baseline 32.64; the original T5 encoder-decoder checkpoint does not follow the few-shot generative answer format (every task near or below chance), so this is not a measurement of ability.

$ where to run it

Other ways to run it

  1. Run it as published · needs ~0.89 GB · from huggingface.co/google-t5
    uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text2text-generation', model='google-t5/t5-base')('translate English to German: The house is wonderful.', max_new_tokens=40)[0]['generated_text'])"

Weights and documentation: huggingface.co/google-t5 ↗

$ how the score was worked out · COPEN — EMNLP 2022 original z… 66%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
COPEN — EMNLP 2022 original zero-shot evaluations3▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 66%CC BY 4.0
prior · pull toward the middle▇▇▇▇▇▇▇▇▇▇ 34%
weighted average= 45.9

not counted (1)

Open Portuguese LLM Leaderboard (archived)Open PT-LLM Leaderboard average (9 Portuguese tasks) [correct]13.7%failed run: Failed run: average 13.68 vs the leaderboard's random baseline 32.64; the original T5 encoder-decoder checkpoint does not follow the few-shot generative answer format (every task near or below chance), so this is not a measurement of ability.

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json