whatmodel.sh / LaMDA 137B (pre-trained, LaMDA-PT)
LaMDA 137B (pre-trained, LaMDA-PT)
↗ share
+ contribute
LaMDA 137B (pre-trained, LaMDA-PT) is a closed-weights language model from Google, released January 2022. Its published size is 137B parameters. Licence: proprietary.
Catalogue identity notes
Google's 137B LaMDA decoder-only language model as used for few-shot prompting in Google's own papers (PaLM paper Table 12 labels it 'Pretraining only'; the self-consistency paper describes LaMDA-137B as 'pre-trained on a mixture of web documents, dialog data and Wikipedia'). LaMDA paper: arXiv:2201.08239 (Jan 2022); Google first announced LaMDA in May 2021. Never released. Comparison-only.
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of LaMDA 137B (pre-trained, LaMDA-PT), out of 877 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 746 |  Gemma 7B IT | 34±1 | local | ||
| 747 |  Mistral 7B Instruct v0.1 | 34±1 | local | ||
| 748 | LaMDA 137B (pre-trained, LaMDA-PT) | 34±1 | never released | ||
| 749 |  GPT-3 text-davinci-001 | 34±1 | comparison only | ||
| 750 | CodeGemma 2B (v1.0) | 34±1 | local | ||
| more from Google | |||||
| 11 |  Gemini 4 Argon | 80±1 | api $4.00/M | ||
| 26 | Gemini 3.8 Flash | 76±1 | api $1.50/M | ||
| 34 | Gemini 3.7 Flash | 75±1 | api $1.50/M | ||
$ what it's good at
coding█·█······█·█ #4 of 4 * · 1.9% of its score · 1 test
math▁▂▃▆█▇▅▇▃▁▁▁ #433 of 523 * · 13% of its score · 1 test
Beats GPT-3.5 Turbo (0125), DBRX Instruct, Command R Plus (04-2024) here, though they rank higher overall. Trails OPT-1.3B, OPT-2.7B, Gemma 2B (base) here, though they rank lower overall.
other tests12 not in a task · 85% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MAWPS (CoT-paper union) lab | 57.9% ✎ | ▅▃█···▅·▃▃▃▅ beats 8/12 | 23% |
| SVAMP lab +1 | 37.5% ✎ | ▃▅█···▅·▃▃▃▅ beats 8/12 | 18% |
| ASDiv lab +1 | 46.6% ✎ | ▅▃▃▅··▃▃·▃▃█ beats 8/12 | 15% |
| CommonsenseQA lab +1 | 53.6% ✎ | █······▁·▃▁▃ beats 8/12 | 9.3% |
| AQuA-RAT lab +1 | 20.6% ✎ | █▃·▁▁·····▂▁ beats 7/12 | 4.8% |
| ARC Challenge lab | 55.1% ✎ | ▄▁▁▂▁▄█▇▃▂▃▂ beats 98/140 | 3.6% |
| BIG-bench Sports Understanding lab | 59.5% ✎ | █·····▃·▃·▄▁ beats 8/12 | 3.6% |
| BIG-bench Date Understanding lab | 21.5% ✎ | ▃·▅·▅▃▅·▃▃·█ beats 8/12 | 3.0% |
4 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| StrategyQA lab +1 | 62.4% ✎ | █·····▅▃▃▅▃█ beats 9/12 | 2.0% |
| AddSub lab | 52.9% ✎ | █·········██ beats 0/2 | 1.3% |
| MultiArith lab | 51.8% ✎ | █·········██ beats 0/2 | 1.2% |
| ARC Easy lab | 75.3% ✎ | ▁▃▂▄▆█▅▃▁··▁ beats 78/100 | 0.3% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Chain-of-Thought Prompting El… 77%, Self-Consistency Improves Cha… 21%, PaLM: Scaling Language Modeli… 2%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs · lab-reported | 9 | 77% |
| Self-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runs · lab-reported | 10 | 21% |
| PaLM: Scaling Language Modeling with Pathways — authors' own runs · lab-reported | 1 | 2% |
| prior · pull toward the middle | 0% | |
| weighted average | = 34.0 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json