whatmodel.sh / Gopher family 44M

Gopher family 44M

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Gopher family 44M is a closed-weights language model from Google DeepMind, released December 2021. Its published size is 0.044B parameters. Licence: proprietary.

Catalogue identity notes

Smallest siblings of Gopher 280B trained by DeepMind on MassiveText (Rae et al., arXiv:2112.11446, Table 1). Never released. Comparison-only.

Model information sources

Release date ↗

30.7±6.7comparison only: no current run option verifiedfrom 1 benchmark result across 1 source* thinly tested, treat the rank as provisional

Score within the LAMBADA (OpenAI version), last-word accuracy comparison set only.

$ how it compares

Scored LLMs released closest to Gopher family 44M, out of 1182 with a score.

#modelscorewhereRAMreleased
1034LaMDA 137B (pre-trained, LaMDA-PT) Google34±1never released2022-01
1035GPT-3 text-davinci-001 OpenAI34±1comparison only2022-01
1106LaMDA 68B (pre-trained) Google27±1never released2022-01
1156LaMDA 8B (pre-trained) Google12±2never released2022-01
1158GPT-3 text-curie-001 OpenAI12±1comparison only2022-01

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests1 not in a task · 81% of its score
testresultvs the others in its rundifficultycountswhy
LAMBADA (OpenAI version), last-word accuracy lab30.1% ✎█··█·█··█·██ beats 0/5medium · avg 58%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 81%at its level

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Scaling Language Models: Meth… 81%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Scaling Language Models: Methods, Analysis & Insights from Training Gopher — maker's own results · lab-reported1Factual vendor-reported score (not licensed content)
prior · pull toward the middle
weighted average= 30.7

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json