whatmodel.sh / GPT-3 XL (1.3B, Brown et al. 2020 paper model)

GPT-3 XL (1.3B, Brown et al. 2020 paper model)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

GPT-3 XL (1.3B, Brown et al. 2020 paper model) is a closed-weights language model from OpenAI, released May 2020. Its published size is 1.3B parameters. Licence: proprietary.

Catalogue identity notes

The research checkpoint 'GPT-3 XL' of OpenAI's GPT-3 paper (Brown et al., arXiv:2005.14165, Table 2.1: 1.3B parameters, 300B training tokens). Weights never released. Deliberately NOT merged with the API engines (gpt-3-ada/babbage/curie/davinci): OpenAI never officially stated which paper size each API engine is (AI21's Jurassic-1 white paper calls the mapping an estimate and notes the API may be 'a slightly different set of models'). Comparison-only.

Model information sources

Release date ↗

49.1±1.9comparison only: no current run option verifiedfrom 13 benchmark results across 1 source

Score within the ARC Challenge comparison set only.

$ how it compares

Scored LLMs released closest to GPT-3 XL (1.3B, Brown et al. 2020 paper model), out of 1182 with a score, then more from OpenAI.

#modelscorewhereRAMreleased
1063GPT-3 davinci (API base model) OpenAI32±1comparison only2020-06
1116GPT-3 curie (API base model) OpenAI25±1comparison only2020-06
1151GPT-3 babbage (API base model) OpenAI16±1comparison only2020-06
1166GPT-3 ada (API base model) OpenAI8±1comparison only2020-06
1168GPT-2 XL (1.5B) OpenAI7±1local~2.8GB2019-11
more from OpenAI
1GPT-6 Astra OpenAI86±1api $20.00/M2026-09
3GPT-6.1 Sol OpenAI84±1api $4.00/M2026-09
8GPT-6 Sol OpenAI81±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests13 not in a task · 99% of its score
testresultvs the others in its rundifficultycountswhy
TriviaQA lab19.7% ✎█·█·██·███·█ beats 3/7hard · avg 22%▇▇▇▇▇▇▇▇ 21%sharp
HellaSwag lab54.7% ✎█··█·██·████ beats 3/7medium · avg 57%▇▇▇▇▇▇ 15%sharp · at its level
ARC Challenge lab35.5% ✎█···██·█████ beats 3/7very hard · avg 35%▇▇▇▇▇ 12%sharp
WinoGrande lab58.7% ✎█·····█·▄▄·█ beats 3/7hard · avg 59%▇▇▇▇ 10%sharp
Natural Questions (closed book) lab4.4% ✎▄··▄·▄·▄█▄·▄ beats 3/7very hard · avg 4.0%▇▇▇▇ 10%sharp · too hard for it
COPA lab77.0% ✎█··▄█·▄▄···▄ beats 4/7medium · avg 77%▇▇ 5.7%at its level
Story Cloze Test lab73.4% ✎▄··▄·▄▄·█▄·▄ beats 3/7medium · avg 74%▇▇ 5.4%at its level
ARC Easy lab53.8% ✎██··██·███·█ beats 3/7medium · avg 55%▇▇ 4.4%at its level
5 more tests
testresultvs the others in its rundifficultycountswhy
LAMBADA (OpenAI version), last-word accuracy lab63.6% ✎█··█·█·█████ beats 3/7medium · avg 64%▇▇ 4.3%at its level
OpenBookQA lab46.8% ✎▄····▄▄▄▄·▄█ beats 3/7hard · avg 48%▇▇ 3.9%at its level
PIQA lab75.1% ✎█···█·██████ beats 3/7medium · avg 74%▇ 3.6%at its level
DROP lab16.4% ✎▄···▄▄▄▄·▄·█ beats 3/7hard · avg 16%▇ 2.4%typical
BoolQ lab62.4% ✎▃········▃██ beats 4/7hard · avg 61%▇ 1.4%noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Language Models are Few-Shot… 99%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Language Models are Few-Shot Learners — maker's own results · lab-reported13Factual vendor-reported score (not licensed content)
prior · pull toward the middle
weighted average= 49.1

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json