whatmodel.sh / U-PaLM 540B

U-PaLM 540B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

U-PaLM 540B is a closed-weights language model from Google, released October 2022. Its published size is 540B parameters. Licence: proprietary.

Catalogue identity notes

Google's U-PaLM 540B: PaLM 540B continued for a small extra compute budget with the UL2R mixture-of-denoisers objective (Tay et al., arXiv:2210.11399, Oct 2022). Never released. Comparison-only.

Model information sources

Release date ↗

48.0±0.9comparison only: no current run option verifiedfrom 21 benchmark results across 2 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of U-PaLM 540B, out of 877 with a score, then more from Google.

#modelscorewhereRAMreleased
484InternVL2-26B Shanghai AI Laboratory (OpenGVLab)48±2local~55GB2024-07
485Aya Expanse 8B Cohere48±2local~6.4GB2024-10
486U-PaLM 540B Google48±1never released2022-10
487GPT-3.5 Turbo (0125) OpenAI48±1api $0.75/M2024-01
488EuroLLM 9B Instruct UTTER (EuroLLM)48±2local~6.9GB2024-11
more from Google
11Gemini 4 Argon Google80±1api $4.00/M2026-09
26Gemini 3.8 Flash Google76±1api $1.50/M2026-09
34Gemini 3.7 Flash Google75±1api $1.50/M2026-08

see all LLMs · find the ones that fit your machine

$ what it's good at

math▁▂▃▆█▇▅▇▃▁▁▁ #216 of 523 · 27% of its score · 2 tests

Beats MiniMax-M2.1, DeepSeek V3 0324, Qwen3 235B-A22B (Apr 2025, hybrid) here, though they rank higher overall. Trails Mixtral 8x7B v0.1 (base), Phi-4-mini-reasoning 3.8B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
MGSM lab +149.9% ✎▆·▂·▄▂▂▂▂█▄▆ beats 16/18medium · avg 47%▇▇▇▇▇▇▇ 18%sharp · at its level · several runs
GSM8K lab58.5% ✎▂▂▂▂▃▄▅▇██▆▄ beats 99/200easy · avg 67%▇▇▇ 8.4%sharp · at its level · small run
other tests16 not in a task · 73% of its score
testresultvs the others in its rundifficultycountswhy
BIG-Bench Hard lab +149.2% ✎▂▂▂▂·▅·▃▂█▃▃ beats 13/18medium · avg 60%▇▇▇▇▇▇▇ 17%sharp · at its level · several runs
MMLU lab +171.5% ✎▁···▁··▃▁▅▅█ beats 16/18easy · avg 74%▇▇▇▇▇▇ 16%at its level · several runs
CommonsenseQA lab80.1% ✎█·····▂▂▃▂▄▄ beats 36/38easy · avg 83%▇▇▇ 8.4%sharp · at its level · small run
TyDiQA-GoldP (1-shot, EM, 8-language average) lab54.6% ✎▂··▂·▂▄▄·███ beats 14/18medium · avg 61%▇▇ 4.5%at its level
ARC Challenge lab60.3% ✎▄▁▁▂▁▄█▇▃▂▃▂ beats 107/140medium · avg 65%▇▇ 4.3%sharp · at its level · small run
WinoGrande lab82.6% ✎▁▁▁▁▂▁▁▅▇█▃▃ beats 127/131easy · avg 85%▇ 3.5%at its level
Natural Questions (closed book) lab21.7% ✎▂▁▁▁▂▄▄▃▅█▅▁ beats 44/68hard · avg 29%▇ 3.4%at its level
OpenBookQA lab53.6% ✎▅▁▁▁▂▂▃█▃▁▃▁ beats 84/107medium · avg 55%▇ 2.8%at its level · small run
8 more tests
testresultvs the others in its rundifficultycountswhy
TriviaQA lab76.4% ✎▁▁▂▂▄▄▃▆▅█▅▂ beats 74/80easy · avg 79%▇ 2.6%typical
BoolQ lab88.8% ✎▁····▁▂█▆▅▆▄ beats 101/103easy · avg 90%▇ 2.6%typical
HellaSwag lab84.1% ✎▁▁▁▂▂▂▄▆█▇▄▂ beats 123/135easy · avg 87%▇ 2.5%typical
COPA lab96.0% ✎▁·▁▁▁▅▅█▆▂▃▁ beats 64/67very easy · avg 95%▇ 1.4%small run
LAMBADA (OpenAI version), last-word accuracy lab80.5% ✎▁·····▁▁▃▃██ beats 86/86easy · avg 84%▇ 1.1%blunt
Story Cloze Test lab87.0% ✎▃▃▃▂▅█▅▂▃··▂ beats 46/50easy · avg 89%▇ 1.0%blunt · small run
PIQA lab84.1% ✎▁▁▁▂▂▃▅█▅▃▁▁ beats 117/123easy · avg 85%▇ 1.0%blunt · at its level
ARC Easy lab85.9% ✎▁▃▂▄▆█▅▃▃··▁ beats 97/100easy · avg 85%▇ 0.9%blunt · small run

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Transcending Scaling Laws wit… 59%, Scaling Instruction-Finetuned… 41%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Transcending Scaling Laws with 0.1% Extra Compute — maker's own results · lab-reported17Factual vendor-reported score (not licensed content)
Scaling Instruction-Finetuned Language Models — authors' own runs · lab-reported4CC BY 4.0
prior · pull toward the middle
weighted average= 48.0

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json