whatmodel.sh / GPT-3 babbage (API base model)

GPT-3 babbage (API base model)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

GPT-3 babbage (API base model) is a closed-weights language model from OpenAI, released June 2020. Licence: proprietary.

Catalogue identity notes

The original OpenAI API base model 'babbage' (2020-2023), retired by OpenAI in January 2024. Not the 2023 replacement babbage-002/davinci-002 and not the InstructGPT 'text-' models. OpenAI never officially published its parameter count, so params_b is omitted.

18.1±1.3comparison only: no current run option verifiedfrom 13 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of GPT-3 babbage (API base model), out of 828 with a score, then more from OpenAI.

#modelscorewhereRAMreleased
799SmolLM2 360M Instruct Hugging Face19±3local~1.7GB2024-10
800GPT-Neo 2.7B EleutherAI18±1local~13GB2021-03
801GPT-3 babbage (API base model) OpenAI18±1comparison only2020-06
802Pythia 1.4B EleutherAI18±2local~2.4GB2023-02
803BLOOM 3B BigScience18±1local~4GB2022-07
more from OpenAI
1GPT-6 Astra OpenAI87±1api $20.00/M2026-09
3GPT-6.1 Sol OpenAI85±1api $4.00/M2026-09
7GPT-6 Sol OpenAI83±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests9 not in a task · 99% of its score
testresultvs the others in its rundifficultycountswhy
HellaSwag +154.5% ✎▄·█·█··▄▄··▄ beats 3/7very easy · avg 92%▇▇▇▇▇▇▇▇▇ 22%at its level · several runs
WinoGrande +159.0% ✎▄·▄·▄▄▄··█·▄ beats 3/7easy · avg 89%▇▇▇▇▇▇▇▇ 19%several runs
LAMBADA (OpenAI version), last-word accuracy +162.4% ✎█·▄··█··█··▄ beats 4/7very easy · avg 90%▇▇▇▇▇ 12%blunt · at its level · several runs
ARC Challenge27.5% ✎▄····▄···█▄▄ beats 1/5easy · avg 79%▇▇▇▇ 10%sharp · too hard for it
TriviaQA11.5% ✎█···█·███··█ beats 1/5easy · avg 72%▇▇▇ 8.5%typical
PIQA +175.5% ✎█··▃··▃▃·▃·▃ beats 4/7easy · avg 89%▇▇▇ 8.1%blunt · at its level · several runs
SciQ86.6% ✎██····███··█ beats 1/5very easy · avg 98%▇▇▇ 7.8%typical
ARC Easy59.8% ✎▄···▄··█·▄·▄ beats 1/5very easy · avg 90%▇▇▇ 7.3%at its level
1 more test
testresultvs the others in its rundifficultycountswhy
OpenBookQA22.4% ✎▅·······█··▃ beats 0/5medium · avg 52%▇▇ 4.6%too hard for it

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · GPT-NeoX-20B: An Open-Source… 66%, EleutherAI GPT-J-6B model car… 33%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
GPT-NeoX-20B: An Open-Source Autoregressive Language Model — authors' own runs9▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 66%CC BY 4.0
EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs4▇▇▇▇▇▇▇▇▇▇ 33%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 18.1

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json