PaLM 540B

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models — authors' own runs · lab-reported	9	▇▇▇▇▇▇▇▇▇▇▇ 36%	CC BY 4.0
PaLM: Scaling Language Modeling with Pathways — authors' own runs · lab-reported	33	▇▇▇▇▇ 16%	CC BY 4.0
Transcending Scaling Laws with 0.1% Extra Compute — maker's own results · lab-reported	17	▇▇▇▇ 12%	Factual vendor-reported score (not licensed content)
Self-Consistency Improves Chain of Thought Reasoning in Language Models — authors' own runs · lab-reported	10	▇▇▇ 11%	CC BY 4.0
Scaling Instruction-Finetuned Language Models — authors' own runs · lab-reported	4	▇▇▇ 9%	CC BY 4.0
Solving Quantitative Reasoning Problems with Language Models — authors' own runs · lab-reported	4	▇▇ 6%	CC BY 4.0
Universal Self-Adaptive Prompting — authors' own runs · lab-reported	9	▇ 3%	CC BY 4.0
Better Zero-Shot Reasoning with Self-Adaptive Prompting — authors' own runs · lab-reported	3	▇ 3%	CC BY 4.0
Large Language Models Encode Clinical Knowledge — authors' own runs · lab-reported	3	▇ 3%	CC BY 4.0
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them — authors' own runs · lab-reported	1	▇ 1%	CC BY 4.0
WinoDict: Probing language models for in-context word acquisition — authors' own runs · lab-reported	1	▇ 1%	CC BY 4.0
prior · pull toward the middle		▇ 0%
weighted average		= 46.6

not counted (5)