whatmodel.sh / PaLM 62B

PaLM 62B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

PaLM 62B is a closed-weights language model from Google, released April 2022. Its published size is 62.5B parameters. Licence: proprietary.

Catalogue identity notes

Google's original pre-trained PaLM 62B (Chowdhery et al., arXiv:2204.02311; Table 1: 62.50B parameters; 780B training tokens for 8B/540B, 795B for 62B). Not the 62B checkpoint trained to 1.3T tokens ('cont-PaLM'), not PaLM 2. Never released as weights or a public API. Comparison-only.

Model information sources

Release date ↗

39.5±0.5comparison only: no current run option verifiedfrom 74 benchmark results across 10 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of PaLM 62B, out of 877 with a score, then more from Google.

#modelscorewhereRAMreleased
674Granite 3B Code Instruct (2K) IBM40±1local~3.7GB2024-04
675Tulu 2 DPO 13B Allen Institute for AI40±1local~9.8GB2023-11
676PaLM 62B Google40±1never released2022-04
677BioMistral-7B BioMistral39±3local~6.1GB2024-02
678SOLAR 10.7B Instruct v1.0 Upstage39±1local~8.3GB2023-12
more from Google
11Gemini 4 Argon Google80±1api $4.00/M2026-09
26Gemini 3.8 Flash Google76±1api $1.50/M2026-09
34Gemini 3.7 Flash Google75±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

coding█·█······█·█ #3 of 4 * · 2.5% of its score · 2 tests
testresultvs the others in its rundifficultycountswhy
HumanEval lab15.9% ✎▁▁▁▁▃▂▆▆█▅▂▁ beats 27/205medium · avg 45%▇ 1.7%sharp
MBPP lab21.4% ✎▁▃▁▄▅█▆▅▄▁▂▁ beats 3/86medium · avg 49%▇ 0.9%at its level · small run
math▁▂▃▆█▇▅▇▃▁▁▁ #357 of 523 · 18% of its score · 3 tests

Beats Molmo 2 8B, DeepSeek-Coder-V2 Instruct (0724), Llama 3.1 Nemotron 70B Instruct here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 7B, Gemma 7B (base) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K lab +329.9% ✎█·▅▃▃▃·▃·▃▃▅ beats 9/12easy · avg 67%▇▇▇▇▇▇ 16%sharp · at its level · several runs
MGSM lab18.2% ✎▆·▂·▄▂▂▃▂█▄▆ beats 8/18medium · avg 47%▇ 2.1%sharp
Minerva MATH lab4.4% ✎▄··▄··▄·▄··█ beats 1/5hard · avg 32%▇ 0.6%typical
other tests46 not in a task · 79% of its score
testresultvs the others in its rundifficultycountswhy
MAWPS (CoT-paper union) lab80.3% ✎▅▃█···▅·▃▃▃▅ beats 9/12very easy · avg 94%▇▇▇▇ 11%sharp
SVAMP lab46.7% ✎▃▅█···▅·▃▃▃▅ beats 9/12easy · avg 82%▇▇▇▇ 9.3%sharp · at its level
ASDiv lab61.9% ✎▅▃▃▅··▃▃·▃▃█ beats 9/12easy · avg 83%▇▇▇ 7.2%sharp · at its level
CommonsenseQA lab +165.4% ✎█······▁·▁▃▃ beats 9/12easy · avg 87%▇▇ 5.9%sharp · at its level · several runs
AQuA-RAT lab22.4% ✎█▁·▁▃·····▂▁ beats 9/12medium · avg 48%▇▇ 4.5%sharp · too hard for it
MMLU lab +155.1% ✎▁···▁··▃▁▅▅█ beats 7/18easy · avg 74%▇▇ 4.3%at its level · several runs
WinoGrande lab +376.6% ✎▄····▄··▄·██ beats 5/6easy · avg 85%▇ 3.3%at its level · several runs
BIG-bench Sports Understanding lab72.1% ✎█·····▃·▁·▄▁ beats 11/12medium · avg 81%▇ 2.8%sharp · at its level
38 more tests
testresultvs the others in its rundifficultycountswhy
TriviaQA lab +267.3% ✎▁▁▂▂▄▄▃▆▅█▅▂ beats 64/80easy · avg 79%▇ 2.2%at its level · several runs
BIG-Bench Hard lab37.4% ✎▂▂▂▂·▅·▃▂█▃▃ beats 5/18medium · avg 60%▇ 2.1%sharp
BoolQ lab +284.8% ✎▁····▁▂█▆▅▆▄ beats 90/103easy · avg 90%▇ 2.0%at its level · several runs
BIG-bench Date Understanding lab29.8% ✎▃·▅·▅▃▅·▃▃·█ beats 9/12medium · avg 52%▇ 2.0%at its level
Natural Questions (closed book) lab +218.1% ✎▂▁▁▁▂▄▄▃▅█▅▁ beats 40/68hard · avg 32%▇ 2.0%several runs
ARC Challenge lab +152.5% ✎▄▁▁▂▁▄█▇▃▂▃▂ beats 91/140medium · avg 66%▇ 1.6%sharp · at its level · small run · several runs
HellaSwag lab +179.7% ✎▁▁▁▂▂▂▄▆█▇▄▂ beats 109/135easy · avg 87%▇ 1.5%at its level · several runs
MMLU-STEM (Minerva subset) lab39.1% ✎▄······▄▄·▄█ beats 2/5medium · avg 62%▇ 1.4%sharp
MedQA USMLE 4-option (few-shot prompted) lab40.9% ✎▄······▄▄▄·█ beats 2/5medium · avg 58%▇ 1.2%typical
ANLI Round 1 (prompted) lab36.4% ✎█▁▃·▃·▄·▃··▁ beats 13/16hard · avg 51%▇ 1.2%sharp · small run
MedMCQA (few-shot prompted) lab43.4% ✎▄·····▄··█·█ beats 2/5medium · avg 55%▇ 1.1%typical
TyDiQA-GoldP (1-shot, EM, 8-language average) lab40.5% ✎▂··▂·▂▄▄·███ beats 7/18medium · avg 61%▇ 1.1%at its level
StrategyQA lab +158.4% ✎█·····▅▃▃▅▃█ beats 7/12medium · avg 70%▇ 1.0%several runs
COPA lab +193.0% ✎▁·▁▁▁▅▅█▆▃▁▁ beats 61/67very easy · avg 96%▇ 0.9%small run · several runs
PIQA lab +280.5% ✎▁▁▁▂▂▃▅█▅▃▁▁ beats 97/123easy · avg 85%▇ 0.7%blunt · at its level · several runs
WiC (prompted) lab49.4% ✎█·▂▁······▁▁ beats 5/13hard · avg 60%▇ 0.7%sharp · too hard for it · noisy · small run
ANLI Round 2 (prompted) lab37.2% ✎█▅▃▅▃▃▅▃█··▃ beats 13/16hard · avg 45%▇ 0.7%small run
LAMBADA (OpenAI version), last-word accuracy lab +175.4% ✎▁·····▁▁▃▃██ beats 77/86easy · avg 84%▇ 0.7%blunt · several runs
MultiRC (SuperGLUE, prompted, F1a) lab72.1% ✎▃·▃██···▅··▃ beats 8/10very easy · avg 85%▇ 0.6%at its level · small run
ARC Easy lab +175.2% ✎▁▃▂▄▆█▅▃▁··▁ beats 84/100easy · avg 85%▇ 0.6%blunt · at its level · small run · several runs
Story Cloze Test lab +183.5% ✎▃▃▃▂▅█▅▂▁··▂ beats 42/50easy · avg 89%▇ 0.6%blunt · at its level · small run · several runs
RTE (prompted) lab67.9% ✎▅▃·▃··▅·▅▅▃█ beats 12/13medium · avg 75%▇ 0.6%at its level · small run
OCWCourses (Minerva) lab5.9% ✎▄·····▄█··▄▄ beats 1/5hard · avg 15%▇ 0.5%too hard for it
ARCADE New Tasks pass@1 (notebook-context prompting) lab7.2% ✎█·█········█ beats 0/2hard · avg 23%▇ 0.5%small run
SQuAD 2.0 (prompted, F1) lab75.6% ✎▃▃···▃▃█▃▃▃▃ beats 9/10very easy · avg 85%▇ 0.4%at its level · small run
PubMedQA (few-shot prompted) lab57.8% ✎▄·······▄▄▄█ beats 2/5medium · avg 75%▇ 0.4%at its level · noisy
CoQA lab75.7% ✎▂···▃▂▂·▆▅▂█ beats 14/17very easy · avg 88%▇ 0.3%small run
Winograd Schema Challenge (WSC273) lab87.5% ✎█▃·▅▃▃█▅██▅▃ beats 20/21easy · avg 92%▇ 0.3%blunt · small run
SuperGLUE WSC (prompted) lab86.3% ✎█··▁·▃·▄▂▁▅▃ beats 25/26easy · avg 83%▇ 0.3%blunt · at its level · small run
ANLI Round 3 (prompted) lab36.7% ✎▂·▆▂██·▄··▂▂ beats 13/16hard · avg 44%▇ 0.3%small run
OpenBookQA lab50.4% ✎▅▁▁▁▂▂▃█▃▁▃▁ beats 74/107medium · avg 59%▇ 0.3%noisy · small run
RACE-middle lab64.3% ✎▄▄····▄█·███ beats 8/10easy · avg 74%▇ 0.2%blunt · at its level · small run
ReCoRD (SuperGLUE, prompted) lab91.4% ✎▄·▄···▄███▄▄ beats 9/10very easy · avg 95%▇ 0.2%blunt · maxed out · small run
QuAC (prompted, F1) lab40.0% ✎▅··▃▃█··▃▅·▃ beats 8/10medium · avg 53%▇ 0.2%blunt · at its level · small run
DROP lab57.9% ✎▃▅█▅·▃·▆▄▂▁▃ beats 29/44medium · avg 62%▇ 0.2%blunt · at its level · small run
RACE-high lab47.5% ✎▂▄·▂▂▂▂▂▆▄▂█ beats 16/17medium · avg 53%▇ 0.2%blunt · at its level · small run
CommitmentBank (prompted) lab57.1% ✎▂▃·▂█▂▂▂·▃·▂ beats 12/13medium · avg 60%▇ 0.1%blunt · at its level · small run
WebQuestions (closed book) lab11.5% ✎▅▅▃▅▃·▃▃▅█▅▃ beats 15/17very hard · avg 4.3%▇ 0.0%blunt · too hard for it · small run

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Chain-of-Thought Prompting El… 49%, PaLM: Scaling Language Modeli… 19%, Scaling Instruction-Finetuned… 8% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs · lab-reported9CC BY 4.0
PaLM: Scaling Language Modeling with Pathways — authors' own runs · lab-reported33CC BY 4.0
Scaling Instruction-Finetuned Language Models — authors' own runs · lab-reported4CC BY 4.0
Solving Quantitative Reasoning Problems with Language Models — authors' own runs · lab-reported4CC BY 4.0
Transcending Scaling Laws with 0.1% Extra Compute — maker's own results · lab-reported7Factual vendor-reported score (not licensed content)
Universal Self-Adaptive Prompting — authors' own runs · lab-reported9CC BY 4.0
Better Zero-Shot Reasoning with Self-Adaptive Prompting — authors' own runs · lab-reported3CC BY 4.0
Large Language Models Encode Clinical Knowledge — authors' own runs · lab-reported3CC BY 4.0
WinoDict: Probing language models for in-context word acquisition — authors' own runs · lab-reported1CC BY 4.0
Natural Language to Code Generation in Interactive Data Science Notebooks — authors' own runs · lab-reported1CC BY 4.0
prior · pull toward the middle
weighted average= 39.5

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json