whatmodel.sh / Flan-PaLM 62B
Flan-PaLM 62B
↗ share
+ contribute
Flan-PaLM 62B is a closed-weights language model from Google, released October 2022. Its published size is 62.5B parameters. Licence: proprietary.
Catalogue identity notes
Google's instruction-finetuned PaLM (Flan-PaLM 62B; Chung et al., arXiv:2210.11416, Table 2). Never released. Comparison-only.
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of Flan-PaLM 62B, out of 877 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 584 |  Llama 3.2 11B Vision Instruct | 45±2 | local | ||
| 585 |  GPT-3.5 text-davinci-002 | 45±0 | comparison only | ||
| 586 | Flan-PaLM 62B | 45±1 | never released | ||
| 587 |  CogVLM2 Llama3 Chat 19B | 44±3 | local | ||
| 588 |  Qwen2.5 3B Instruct | 44±1 | local | ||
| more from Google | |||||
| 11 |  Gemini 4 Argon | 80±1 | api $4.00/M | ||
| 26 | Gemini 3.8 Flash | 76±1 | api $1.50/M | ||
| 34 | Gemini 3.7 Flash | 75±1 | api $1.50/M | ||
$ what it's good at
math▁▂▃▆█▇▅▇▃▁▁▁ #301 of 523 * · 21% of its score · 1 test
Beats Command A (03-2025), GPT-4o (2024-05-13), GPT-4o (2024-11-20) here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Gemma 7B (base), CodeGemma 7B (v1.0) here, though they rank lower overall.
other tests6 not in a task · 78% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MMLU lab | 59.6% ✎ | ▁···▁··▃▁▅▅█ beats 9/18 | 25% |
| BIG-Bench Hard lab | 47.5% ✎ | ▂▂▂▂·▅·▃▂█▃▃ beats 10/18 | 20% |
| MedQA USMLE 4-option (few-shot prompted) lab | 46.1% ✎ | ▄······▄▄▄·█ beats 3/5 | 11% |
| MedMCQA (few-shot prompted) lab | 46.2% ✎ | ▄·····▄··█·█ beats 3/5 | 10% |
| TyDiQA-GoldP (1-shot, EM, 8-language average) lab | 58.7% ✎ | ▂··▂·▂▄▄·███ beats 15/18 | 8.8% |
| PubMedQA (few-shot prompted) lab | 77.2% ✎ | ▄·······▄▄▄█ beats 4/5 | 3.2% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Scaling Instruction-Finetuned… 75%, Large Language Models Encode… 24%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Scaling Instruction-Finetuned Language Models — authors' own runs · lab-reported | 4 | 75% |
| Large Language Models Encode Clinical Knowledge — authors' own runs · lab-reported | 3 | 24% |
| prior · pull toward the middle | 1% | |
| weighted average | = 44.5 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json