whatmodel.sh / Minerva 8B
Minerva 8B
↗ share
+ contribute
Minerva 8B is a closed-weights language model from Google, released June 2022. Its published size is 8.63B parameters. Licence: proprietary.
Catalogue identity notes
Google's Minerva 8B: PaLM further trained on mathematical web pages and arXiv (Lewkowycz et al., arXiv:2206.14858, Table 2). Never released. Comparison-only.
Model information sources
Score within the LMArena (text) comparison set only.
$ how it compares
The LLMs ranked either side of Minerva 8B, out of 877 with a score, then more from Google.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 735 |  Granite 3B Code Base (2K) | 35±1 | local | ||
| 736 |  StarCoderBase 15.5B | 35±1 | local | ||
| 737 | Minerva 8B | 35±2 | never released | ||
| 738 |  OLMo 2 1B Instruct (0425) | 35±1 | local | ||
| 739 |  ChatGLM3 6B 128K | 35±4 | local | ||
| more from Google | |||||
| 11 |  Gemini 4 Argon | 80±1 | api $4.00/M | ||
| 26 | Gemini 3.8 Flash | 76±1 | api $1.50/M | ||
| 34 | Gemini 3.7 Flash | 75±1 | api $1.50/M | ||
$ what it's good at
math▁▂▃▆█▇▅▇▃▁▁▁ #413 of 523 * · 68% of its score · 2 tests
Beats Athene-70B, Gemma 2 9B IT, GPT-3.5 Turbo (0125) here, though they rank higher overall. Trails OPT-2.7B, Gemma 2B (base), Stable Code 3B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GSM8K lab | 16.2% ✎ | ▃····▃··▃··█ beats 1/5 | 59% |
| Minerva MATH lab | 14.1% ✎ | ▄··▄··▄·▄··█ beats 3/5 | 8.4% |
other tests2 not in a task · 30% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MMLU-STEM (Minerva subset) lab | 35.6% ✎ | ▄······▄▄·▄█ beats 1/5 | 22% |
| OCWCourses (Minerva) lab | 7.7% ✎ | ▄·····▄█··▄▄ beats 3/5 | 8.2% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Solving Quantitative Reasonin… 98%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Solving Quantitative Reasoning Problems with Language Models — authors' own runs · lab-reported | 4 | 98% |
| prior · pull toward the middle | 2% | |
| weighted average | = 35.0 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json