whatmodel.sh / OPT-13B (13 billion parameters)
OPT-13B (13 billion parameters)
↗ share
+ contribute
OPT-13B (13 billion parameters) is an open-weights language model from Meta, released May 2022. Its published size is 13B parameters and a 2k-token context window. Licence: opt-175b-license-noncommercial.
Catalogue identity notes
Meta's Open Pre-trained Transformer base model as released on Hugging Face (weights under Meta's non-commercial OPT licence).
$ how it compares
The LLMs ranked either side of OPT-13B (13 billion parameters), out of 828 with a score, then more from Meta.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 769 |  GPT-3 text-davinci-001 | 26±3 | comparison only | ||
| 770 |  GPT4All-13B-Snoozy | 26±2 | local | ||
| 771 | OPT-13B (13 billion parameters) | 26±1 | local | ||
| 772 |  Granite 4.0 350M | 26±1 | local | ||
| 773 | GPT-3 curie (API base model) | 26±1 | comparison only | ||
| more from Meta | |||||
| 24 | Muse Spark 1.3 | 77±1 | api $2.00/M | ||
| 40 | Muse Spark 1.2 | 75±1 | api $2.00/M | ||
| 43 | Muse Spark 1.1 | 75±1 | api $2.00/M | ||
$ what it's good at
other tests11 not in a task · 99% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| ARC Challenge | 32.9% ✎ | █··▁·▁▁·▁▂▃▂ beats 17/19 | 20% |
| WinoGrande | 65.2% ✎ | ▄····▄▆▂▄▆▆█ beats 16/19 | 17% |
| ARC Easy | 67.1% ✎ | ▃·▅▃▃▅·██▃██ beats 15/19 | 17% |
| Natural Questions (closed book) | 13.5% ✎ | ▁·▁·▁▂▁▂·▅▂█ beats 10/28 | 14% |
| PIQA | 76.0% ✎ | ▂·▄▄·▄▂▆▂▄█▄ beats 15/19 | 9.4% |
| SciQ | 90.8% ✎ | ▂··▂▃▂▂▆▃▃█▂ beats 17/19 | 8.5% |
| LAMBADA (OpenAI version), last-word accuracy | 68.6% ✎ | ▃··▅▃▃▅▅▅███ beats 16/19 | 7.0% |
| TriviaQA | 41.0% ✎ | ▁··▁█▃▃▁▄▅▄█ beats 16/28 | 2.3% |
3 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Open PT-LLM Leaderboard average (9 Portuguese tasks) | 28.2% ✎ | ▃▁▁▁▁▁▁▁▂█▅▁ beats 15/276 | 1.9% |
| TruthfulQA MC2 | 34.0% ✎ | ▅▃█▆▁▂▅···▁▁ beats 1/31 | 1.2% |
| HaluEval QA hallucination detection (accuracy) | 35.3% ✎ | █▁▁·▁▁·▁▁▁▁▁ beats 8/27 | 0.7% |
$ where to run it
Other ways to run it
- Run it as published · needs ~25.71 GB · from huggingface.co/facebook
uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='facebook/opt-13b', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"
Weights and documentation: huggingface.co/facebook ↗
$ how the score was worked out · Pythia: A Suite for Analyzing… 79%, Hallucinations Leaderboard re… 18%, Open Portuguese LLM Leaderboa… 2%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling — authors' own runs | 6 | 79% |
 Hallucinations Leaderboard results (HF dataset hallucinations-leaderboard/results) | 4 | 18% |
| Open Portuguese LLM Leaderboard (archived) | 1 | 2% |
| prior · pull toward the middle | 1% | |
| weighted average | = 26.1 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json