whatmodel.sh / Gopher family 7.1B

Gopher family 7.1B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Gopher family 7.1B is a closed-weights language model from Google DeepMind, released December 2021. Its published size is 7.1B parameters. Licence: proprietary.

Catalogue identity notes

Smaller sibling of Gopher 280B trained by DeepMind on MassiveText (Rae et al., arXiv:2112.11446, Table 1). Never released. Comparison-only.

Model information sources

Release date ↗

50.8±3.6comparison only: no current run option verifiedfrom 2 benchmark results across 1 source* thinly tested, treat the rank as provisional

Score within the LAMBADA (OpenAI version), last-word accuracy comparison set only.

$ how it compares

Scored LLMs released closest to Gopher family 7.1B, out of 1182 with a score.

#modelscorewhereRAMreleased
1034LaMDA 137B (pre-trained, LaMDA-PT) Google34±1never released2022-01
1035GPT-3 text-davinci-001 OpenAI34±1comparison only2022-01
1106LaMDA 68B (pre-trained) Google27±1never released2022-01
1156LaMDA 8B (pre-trained) Google12±2never released2022-01
1158GPT-3 text-curie-001 OpenAI12±1comparison only2022-01

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests2 not in a task · 95% of its score
testresultvs the others in its rundifficultycountswhy
MMLU lab29.5% ✎▁▁▁▁▃▂▂▅██▆▄ beats 13/214very hard · avg 30%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 70%sharp · too hard for it
LAMBADA (OpenAI version), last-word accuracy lab68.5% ✎█··█·█··█·██ beats 4/5medium · avg 58%▇▇▇▇▇▇▇▇▇▇ 25%at its level

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Scaling Language Models: Meth… 95%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Scaling Language Models: Methods, Analysis & Insights from Training Gopher — maker's own results · lab-reported2Factual vendor-reported score (not licensed content)
prior · pull toward the middle
weighted average= 50.8

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json