whatmodel.sh / MPT-7B (base)
MPT-7B (base)
↗ share
+ contribute
MPT-7B (base) is an open-weights language model from MosaicML, released May 2023. Its published size is 6.7B parameters. Licence: apache-2.0.
Catalogue identity notes
MosaicML's pretrained MPT base model (not -instruct/-chat). The official mosaicml/mpt-* repositories no longer resolve on Hugging Face (HTTP 401 on 2026-09-28), so no run command is listed.
$ how it compares
The LLMs ranked either side of MPT-7B (base), out of 828 with a score, then more from MosaicML.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 742 |  Baichuan2-7B-Base | 30±1 | local | ||
| 743 |  LLaMA 7B | 30±1 | comparison only | ||
| 744 | MPT-7B (base) | 30±1 | comparison only | ||
| 745 |  OLMo 7B Instruct | 29±2 | local | ||
| 746 |  StableLM-3B-4E1T | 29±1 | local | ||
| more from MosaicML | |||||
| 693 | MPT-30B-Chat | 35±2 | comparison only | ||
| 765 | MPT-7B-Chat | 27±2 | comparison only | ||
$ what it's good at
other tests14 not in a task · 100% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| ARC Challenge +2 | 41.9% ✎ | ▁·▁··▁▁▇▂█▆▃ beats 16/26 | 36% |
| WinoGrande +2 | 68.0% ✎ | ▁····▁··▁▂█▄ beats 19/26 | 14% |
| BoolQ +2 | 73.9% ✎ | ▂▃▂▅▆▂▅▆█▂▂▂ beats 19/26 | 10% |
| HellaSwag +2 | 76.2% ✎ | ▃···▁▇▅█▇▃▁▁ beats 20/26 | 7.7% |
| ARC Easy +2 | 70.0% ✎ | ▁▁·▁▁▄▅▄█▅▁▃ beats 19/26 | 7.6% |
| OpenBookQA +2 | 31.4% ✎ | █·▁▁▃·▁▃▅▇▃▄ beats 18/26 | 7.0% |
| SciQ +1 | 93.7% ✎ | ▁·▁·▁·▃▅▃█▄▁ beats 18/26 | 4.6% |
| PIQA +2 | 78.9% ✎ | ▁·▁··▁▁▃█▅█▂ beats 21/26 | 4.6% |
6 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Social IQA +1 | 45.1% ✎ | ▁···▁▃█▁▃▁·▁ beats 23/26 | 2.7% |
| Factual knowledge recall, Premium2k exact match (zero-shot) | 3.5% ✎ | ▅····▃▅·██·█ beats 1/13 | 2.2% |
| Factual knowledge recall, Premium2k exact match (10-shot) | 30.4% ✎ | ▃▅▃··▅▃·█▃▃▅ beats 4/13 | 1.9% |
| LAMBADA (OpenAI version), last-word accuracy | 68.6% ✎ | ▁··▂▁▁▁▂▆█▅▃ beats 12/26 | 1.3% |
| TruthfulQA MC2 | 33.5% ✎ | █·█·█▄···▄·█ beats 1/9 | 0.3% |
| COPA | 65.6% ✎ | ▅·▃·▃··█▃··▃ beats 3/8 | 0.2% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · Stability AI — original Stabl… 51%, OLMo: Accelerating the Scienc… 24%, Textbooks Are All You Need II… 20% and 2 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Stability AI — original StableLM zero-shot evaluation outputs | 11 | 51% |
 OLMo: Accelerating the Science of Language Models — authors' own runs | 8 | 24% |
| Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs | 8 | 20% |
| Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1) | 2 | 4% |
| From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items (arXiv:2410.14897v1) | 1 | 0% |
| prior · pull toward the middle | 0% | |
| weighted average | = 29.5 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json