whatmodel.sh / Minerva 540B
Minerva 540B
↗ share
+ contribute
Minerva 540B is a closed-weights language model from Google, released June 2022. Its published size is 540.35B parameters. Licence: proprietary.
Catalogue identity notes
Google's Minerva 540B: PaLM further trained on mathematical web pages and arXiv (Lewkowycz et al., arXiv:2206.14858, Table 2). Never released. Comparison-only.
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of Minerva 540B, out of 877 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 490 |  GPT-3.5 Turbo (1106) | 48±1 | comparison only | ||
| 491 |  Granite 4.0 H Micro 3B | 48±1 | local | ||
| 492 | Minerva 540B | 48±1 | never released | ||
| 493 |  Code Llama 70B Instruct | 48±1 | local | ||
| 494 |  OLMo 2 13B Instruct (1124) | 48±1 | local | ||
| more from Google | |||||
| 11 |  Gemini 4 Argon | 80±1 | api $4.00/M | ||
| 26 | Gemini 3.8 Flash | 76±1 | api $1.50/M | ||
| 34 | Gemini 3.7 Flash | 75±1 | api $1.50/M | ||
$ what it's good at
math▁▂▃▆█▇▅▇▃▁▁▁ #241 of 523 · 54% of its score · 2 tests
Beats MiniMax-M2.1, DeepSeek V3 (Dec '24), GPT-4o (2024-08-06) here, though they rank higher overall. Trails Phi-2 (2.7B), Granite 8B Code Base (4K), Mixtral 8x7B v0.1 (base) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GSM8K lab +1 | 58.8% ✎ | ▃····▃··▃··█ beats 5/5 | 48% |
| Minerva MATH lab | 33.6% ✎ | ▄··▄··▄·▄··█ beats 5/5 | 6.2% |
other tests4 not in a task · 46% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| CommonsenseQA lab | 72.2% ✎ | █·····▂▂▃▂▄▄ beats 32/38 | 17% |
| MMLU-STEM (Minerva subset) lab | 63.9% ✎ | ▄······▄▄·▄█ beats 5/5 | 13% |
| BIG-Bench Hard lab | 37.2% ✎ | ▁▁▁▁▂▄▅▆█▃▃▁ beats 28/155 | 10% |
| OCWCourses (Minerva) lab | 17.6% ✎ | ▄·····▄█··▄▄ beats 5/5 | 5.8% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Solving Quantitative Reasonin… 56%, Transcending Scaling Laws wit… 43%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Solving Quantitative Reasoning Problems with Language Models — authors' own runs · lab-reported | 4 | 56% |
| Transcending Scaling Laws with 0.1% Extra Compute — maker's own results · lab-reported | 3 | 43% |
| prior · pull toward the middle | 1% | |
| weighted average | = 47.7 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json