whatmodel.sh / GLaM (64B/64E, 1.2T MoE)

GLaM (64B/64E, 1.2T MoE)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

GLaM (64B/64E, 1.2T MoE) is a closed-weights language model from Google, released December 2021. Its published size is 1200B parameters (96.6B active per token). Licence: proprietary.

Catalogue identity notes

Google's GLaM 64B/64E mixture-of-experts model (Du et al., arXiv:2112.06905, Table 4: 1200B total / 96.6B activated parameters), trained on the GLaM dataset. The flagship GLaM model. Never released. Not the same checkpoint as LaMDA-PT even where sizes match. Comparison-only.

Model information sources

Release date ↗

68.0±1.2comparison only: no current run option verifiedfrom 29 benchmark results across 1 source

Score within the ANLI Round 1 (prompted) comparison set only.

$ how it compares

Scored LLMs released closest to GLaM (64B/64E, 1.2T MoE), out of 1182 with a score, then more from Google.

#modelscorewhereRAMreleased
1034LaMDA 137B (pre-trained, LaMDA-PT) Google34±1never released2022-01
1035GPT-3 text-davinci-001 OpenAI34±1comparison only2022-01
1106LaMDA 68B (pre-trained) Google27±1never released2022-01
1156LaMDA 8B (pre-trained) Google12±2never released2022-01
1158GPT-3 text-curie-001 OpenAI12±1comparison only2022-01
more from Google
11Gemini 4 Argon Google80±1api $4.00/M2026-09
26Gemini 3.8 Flash Google76±1api $1.50/M2026-09
34Gemini 3.7 Flash Google75±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests29 not in a task · 99% of its score
testresultvs the others in its rundifficultycountswhy
Natural Questions (closed book) lab24.7% ✎▄·▄··▄·█▄·▄▄ beats 7/7very hard · avg 7.2%▇▇▇▇▇ 14%sharp
TriviaQA lab71.3% ✎▄··▄···▄·█▄█ beats 7/7hard · avg 35%▇▇▇▇▇ 12%sharp · at its level
ARC Challenge lab48.0% ✎▂··▂·····▄·█ beats 7/7very hard · avg 35%▇▇▇▇ 10%sharp · at its level
WebQuestions (closed book) lab19.0% ✎▄····▄·▄█▄▄▄ beats 7/7very hard · avg 6.7%▇▇▇ 8.3%sharp
WinoGrande lab73.5% ✎▃··▃·····▃▅█ beats 7/7hard · avg 61%▇▇▇ 7.9%sharp · at its level
SQuAD 2.0 (prompted, F1) lab71.1% ✎██····██████ beats 7/7medium · avg 47%▇▇ 5.2%sharp · at its level
HellaSwag lab76.6% ✎▃··▃···▃·▃▃█ beats 6/7medium · avg 62%▇▇ 5.0%sharp · at its level
RACE-high lab46.9% ✎▃▃······▅·▃█ beats 6/7hard · avg 39%▇▇ 4.5%at its level
21 more tests
testresultvs the others in its rundifficultycountswhy
BoolQ lab83.1% ✎█···▃··▃▃·▃▃ beats 7/7hard · avg 65%▇ 3.2%sharp · at its level · noisy
COPA lab90.0% ✎▄·▄···▄·▄█·█ beats 7/7easy · avg 83%▇ 2.7%typical
Story Cloze Test lab82.5% ✎▄·▄····▄▄▄▄█ beats 7/7medium · avg 75%▇ 2.6%at its level
RACE-middle lab64.0% ✎▄▄····▄▄·▄█▄ beats 6/7medium · avg 56%▇ 2.5%at its level
Winograd Schema Challenge (WSC273) lab87.2% ✎█·····▄▄▄▄▄▄ beats 7/7medium · avg 79%▇ 2.5%at its level
SuperGLUE WSC (prompted) lab85.3% ✎█·····▄·▄█▄▄ beats 7/7medium · avg 77%▇ 2.5%at its level
DROP lab57.3% ✎▄·▄██▄·····▄ beats 7/7hard · avg 16%▇ 2.4%sharp · at its level · noisy
ARC Easy lab71.6% ✎▄··▄···▄·█▄█ beats 6/7medium · avg 63%▇ 2.1%at its level
RTE (prompted) lab67.9% ✎█▄··▄····▄▄█ beats 7/7very hard · avg 55%▇ 2.1%sharp · at its level · noisy
OpenBookQA lab53.4% ✎▄·▄····█·█·█ beats 7/7hard · avg 46%▇ 1.8%at its level
CoQA lab78.8% ✎▃▃·····█▃▃·▃ beats 7/7medium · avg 64%▇ 1.7%typical
PIQA lab80.4% ✎▃···▃··▃·▃█▃ beats 7/7medium · avg 75%▇ 1.6%at its level
LAMBADA (OpenAI version), last-word accuracy lab64.2% ✎▄▄·····▄·█▄█ beats 4/7medium · avg 60%▇ 1.4%at its level
ANLI Round 3 (prompted) lab41.3% ✎▄·▄▄██·····▄ beats 7/7very hard · avg 35%▇ 1.3%sharp · too hard for it · noisy
ReCoRD (SuperGLUE, prompted) lab90.3% ✎▄·▄·····▄▄██ beats 7/7very easy · avg 86%▇ 1.2%maxed out
QuAC (prompted, F1) lab40.3% ✎█···▄·█▄··▄▄ beats 7/7hard · avg 32%▇ 0.7%blunt · at its level
ANLI Round 1 (prompted) lab39.2% ✎█·▄▄·▄··▄··█ beats 6/7very hard · avg 35%▇ 0.4%too hard for it · noisy
ANLI Round 2 (prompted) lab37.3% ✎█▄·····█·▄·█ beats 7/7very hard · avg 35%▇ 0.1%blunt · too hard for it · noisy
MultiRC (SuperGLUE, prompted, F1a) lab73.7% ✎▃···▅▃█····▃ beats 7/7medium · avg 56%▇ 0.0%blunt · at its level · noisy
CommitmentBank (prompted) lab48.2% ✎█▄▄█·▄·····▄ beats 6/7medium · avg 43%▇ 0.0%blunt · at its level · noisy
WiC (prompted) lab50.3% ✎█▄·······▂·▂ beats 5/7very hard · avg 50%▇ 0.0%blunt · too hard for it · noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · GLaM: Efficient Scaling of La… 99%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts — authors' own runs · lab-reported29CC BY 4.0
prior · pull toward the middle
weighted average= 68.0

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json