whatmodel.sh / GPT-1 (openai-gpt)

GPT-1 (openai-gpt)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

GPT-1 (openai-gpt) is an open-weights language model from OpenAI, released June 2018. Its published size is 0.117B parameters and a 1k-token context window. Licence: mit.

Catalogue identity notes

The original 12-layer, 512-token-context GPT (Radford et al. 2018), as hosted in openai-community/openai-gpt. Size from OpenAI's GPT-2 paper (117M; same as the smallest GPT-2 configuration).

2.7±4.7#943 of 949 LLMsfrom 1 benchmark result across 1 source* thinly tested, treat the rank as provisional

$ how it compares

The LLMs ranked either side of GPT-1 (openai-gpt), out of 1132 with a score, then more from OpenAI.

#modelscorewhereRAMreleased
1124OPT-350M Meta5±2local~2.2GB2022-05
1125StableLM-Base-Alpha 3B Stability AI4±1local~17GB2023-04
1126GPT-1 (openai-gpt) OpenAI3±5local~2GB2018-06
1127Pythia 160M EleutherAI3±2local~1.6GB2023-02
1128OPT-125M Meta3±2local~1.8GB2022-05
more from OpenAI
1GPT-6 Astra OpenAI87±1api $20.00/M2026-09
3GPT-6.1 Sol OpenAI85±1api $4.00/M2026-09
7GPT-6 Sol OpenAI83±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests2 not in a task · 90% of its score
testresultvs the others in its rundifficultycountswhy
PROST physical reasoning (zero-shot, 10-concept macro average)26.1% ✎█···██·█···█ beats 1/4easy · avg 79%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 90%too hard for it
Open PT-LLM Leaderboard average (9 Portuguese tasks)2.0% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 0/276medium · avg 73%—failed run: Already fixed by chance 0.3264 (c7b3953). 1.96% average vs a 32.64% random baseline; GPT-1's 512-token context cannot hold 3- to 25-shot prompts.

$ where to run it

Other ways to run it

  1. Run it as published · needs ~0.48 GB · from huggingface.co/openai-community
    uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='openai-community/openai-gpt')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Weights and documentation: huggingface.co/openai-community ↗

$ how the score was worked out · PROST (Aroca-Ouellette et al.… 90%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
PROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runs1▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 90%CC BY 4.0
prior · pull toward the middle▇▇▇ 10%
weighted average= 2.7

not counted (1)

Open Portuguese LLM Leaderboard (archived)Open PT-LLM Leaderboard average (9 Portuguese tasks) [correct]2.0%failed run: Already fixed by chance 0.3264 (c7b3953). 1.96% average vs a 32.64% random baseline; GPT-1's 512-token context cannot hold 3- to 25-shot prompts.

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json