whatmodel.sh / OPT-175B

OPT-175B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

OPT-175B is an open-weights language model from Meta, released May 2022. Its published size is 175B parameters. Licence: opt-175b-license-noncommercial.

Catalogue identity notes

Meta OPT-175B pretrained base model; weights released on request to researchers under the non-commercial OPT-175B licence (not on the public Hugging Face hub, so no run command). Not OPT-IML. arXiv:2205.01068v1 (submitted 2 May 2022) abstract: 'a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters, which we aim to fully and responsibly share with interested researchers.'

Model information sources

Release date ↗

32.8±1.0comparison only: no current run option verifiedfrom 26 benchmark results across 4 sources

Score within the LMArena (text) comparison set only.

$ how it compares

The LLMs ranked either side of OPT-175B, out of 877 with a score, then more from Meta.

#modelscorewhereRAMreleased
756Qwen-7B (base) Alibaba33±1local~6.6GB2023-08
757LFM2 350M Liquid AI33±1local~1.7GB2025-07
758OPT-175B Meta33±1comparison only2022-05
759BLOOMZ 176B BigScience33±2comparison only2022-11
760Codex code-davinci-001 OpenAI33±1comparison only2021-08
more from Meta
5Muse Wonder Full Meta82±3Muse Code only2026-10
16Muse Wonder Meta78±2api $8.00/M2026-10
24Muse Spark 1.3 Meta76±1api $2.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests18 not in a task · 100% of its score
testresultvs the others in its rundifficultycountswhy
ARC Challenge +243.9% ✎█▃··▃··▅▃█▅▅ beats 13/14easy · avg 81%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 39%sharp · at its level · several runs
WinoGrande +172.5% ✎▃·▃▃▃▅·█···▃ beats 7/9easy · avg 87%▇▇▇▇▇ 12%at its level · several runs
ARC Easy +271.0% ✎▅▃▃▃▃▃▃▃▅·█▃ beats 14/14easy · avg 84%▇▇▇ 8.0%blunt · at its level · several runs
Story Cloze Test +279.8% ✎▂▂▄▂·▂·▆·▂█▃ beats 14/14easy · avg 89%▇▇▇ 7.7%blunt · at its level · several runs
MMLU lab34.2% ✎▁▁▁▁▁▃▂▅██▆▄ beats 18/214medium · avg 59%▇▇ 4.7%small run
BIG-Bench Hard lab30.2% ✎▁▁▁▃▂▄▅▆█▃▃▁ beats 11/155medium · avg 59%▇▇ 4.6%sharp · noisy · small run
PIQA +181.1% ✎▄·█▄·█▄▄▄███ beats 14/14easy · avg 88%▇▇ 4.0%blunt · at its level · several runs
VEC embodied concepts, zero-shot perplexity probing (average)55.8% ✎▁····▁▁·▁·▃█ beats 8/11hard · avg 63%▇ 3.7%typical
10 more tests
testresultvs the others in its rundifficultycountswhy
RTE (prompted) lab60.3% ✎▅▃·▃··▅·▅▅▃█ beats 8/13medium · avg 75%▇ 3.2%small run
WiC (prompted) lab56.6% ✎█·▂▁······▃▁ beats 12/13medium · avg 71%▇ 2.9%sharp · too hard for it · noisy · small run
LAMBADA (OpenAI version), last-word accuracy75.6% ✎▄▄▄█▄·█·▄███ beats 14/14very easy · avg 87%▇ 2.4%blunt · at its level
COPA lab87.0% ✎▁·▁▁▁▅▅█▆▂▁▁ beats 50/67very easy · avg 95%▇ 2.0%small run
OpenBookQA lab58.4% ✎▅▁▁▁▂▂▃█▃▁▃▁ beats 90/107easy · avg 75%▇ 1.8%at its level · noisy · small run
BoolQ lab60.1% ✎▁····▁▂█▆▅▆▄ beats 12/103easy · avg 84%▇ 1.7%at its level · small run
Winograd Schema Challenge (WSC273) lab85.3% ✎█▃·▅▃▃█▅██▅▃ beats 16/21easy · avg 92%▇ 1.4%blunt · at its level · small run
CommitmentBank (prompted) lab46.4% ✎▂▃·▂█▃▂▂·▂·▂ beats 9/13medium · avg 60%▇ 0.4%blunt · at its level · small run
SuperGLUE WSC (prompted) lab51.4% ✎█··▃·▃·▄▂▁▅▃ beats 8/26medium · avg 70%▇ 0.4%blunt · noisy · small run
VEC visual concepts, zero-shot perplexity probing (average)69.9% ✎▅▅▃█·▃··▃·▃▃ beats 11/11hard · avg 66%▇ 0.1%blunt · at its level

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · GPTQ: Accurate Post-Training… 42%, OPT-IML: Scaling Language Mod… 35%, Outlier Suppression+: Accurat… 19% and 1 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — authors' own runs5CC BY 4.0
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization — authors' own runs · lab-reported15CC BY 4.0
Outlier Suppression+: Accurate quantization of large language models by equivalent and effective shifting and scaling — authors' own runs4CC BY 4.0
Can Language Models Understand Physical Concepts? — authors' own runs2CC BY 4.0
prior · pull toward the middle
weighted average= 32.8

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json