whatmodel.sh / GPT-2 Small (124M)

GPT-2 Small (124M)

+ contribute

GPT-2 Small (124M) is an open-weights language model from OpenAI, released February 2019. Its published size is 0.124B parameters and a 1k-token context window. Licence: mit.

Catalogue identity notes

OpenAI's released GPT-2 checkpoint as hosted at openai-community/gpt2 (formerly the root-namespace 'gpt2' repo). Base (pretrained) model; parameter count as corrected in openai/gpt-2 README.

-9.5±2.5#719 of 719 LLMsfrom 14 benchmark results across 5 sources

$ how it compares

The LLMs ranked either side of GPT-2 Small (124M), out of 828 with a score, then more from OpenAI.

#	model	score	where	RAM	released
824	OPT-125M Meta	3±2	local	~1.8GB	2022-05
825	GPT-2 Large (774M) OpenAI	1±2	local	~2GB	2019-08
826	GPT-2 Medium (355M) OpenAI	-1±2	local	~1.8GB	2019-05
827	Pythia 70M EleutherAI	-7±2	local	~1.6GB	2023-02
828	GPT-2 Small (124M) OpenAI	-9±3	local	~1.6GB	2019-02
	more from OpenAI
1	GPT-6 Astra OpenAI	87±1	api $20.00/M		2026-09
3	GPT-6.1 Sol OpenAI	85±1	api $4.00/M		2026-09
7	GPT-6 Sol OpenAI	83±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests14 not in a task · 97% of its score

test	result	vs the others in its run	difficulty	counts	why
OpenBookQA	16.4% ✎	█▁▁▁▄▃▃▇▂▁▄▂ beats 8/65	very hard · avg 26%	▇▇▇▇▇▇▇▇▇ 22%	too hard for it
HellaSwag	28.6% ✎	▃▁▁▁▂▂▃▇█▆▅▃ beats 0/85	easy · avg 88%	▇▇▇▇▇ 13%	too hard for it
ARC Easy	42.9% ✎	▁▃▃▂▃▅▇█▇▆▄▆ beats 2/65	very easy · avg 92%	▇▇▇▇▇ 11%	typical
CommonsenseQA	25.5% ✎	█·····█·█··█ beats 0/3	easy · avg 83%	▇▇▇▇ 11%	typical
COPA	61.0% ✎	▄·▄·█··▆▂··▂ beats 3/12	very easy · avg 96%	▇▇▇▇ 10%	typical
ARC Challenge	20.1% ✎	▆▁▁▂▂▅▇█▄▃▃▂ beats 2/97	hard · avg 46%	▇▇▇▇ 9.4%	sharp · too hard for it
PROST physical reasoning (zero-shot, 10-concept macro average)	23.6% ✎	█···██·█···█ beats 0/4	easy · avg 79%	▇▇▇▇ 9.3%	too hard for it
Story Cloze Test	60.0% ✎	█····█···█·█ beats 0/3	easy · avg 92%	▇▇▇ 8.4%	blunt

6 more tests

test	result	vs the others in its run	difficulty	counts	why
COPEN concept identification in context (zero-shot, full prompt)	31.1% ✎	▃···▃▃·▃·▃·█ beats 1/7	hard · avg 44%	▇ 1.6%	blunt · too hard for it
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy	22.8% ✎	▃▁▁▁▁▁▁▂▄█▄▃ beats 0/146	easy · avg 83%	▇ 0.6%	too hard for it · noisy
Open PT-LLM Leaderboard average (9 Portuguese tasks)	23.3% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 0/276	medium · avg 73%	▇ 0.6%	too hard for it · noisy
COPEN conceptual similarity judgment (zero-shot, full prompt)	7.9% ✎	▄···█···██·▄ beats 2/7	very hard · avg 11%	▇ 0.1%	blunt · too hard for it
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy	24.7% ✎	▃·▁▁▂▁▂▅█▇▄▁ beats 4/100	medium · avg 68%	▇ 0.1%	too hard for it · noisy
COPEN conceptual property judgment (zero-shot, full prompt)	51.5% ✎	█·······▃··▁ beats 6/7	very hard · avg 51%	▇ 0.1%	blunt · too hard for it · noisy

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run GPT-2 Small (124M) · needs ~0.1 GB · from huggingface.co/mradermacher
chat in the terminal
llama cli -hf mradermacher/gpt2-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf mradermacher/gpt2-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run GPT-2 Small (124M) · needs ~0.1 GB · from huggingface.co/mradermacher
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/mradermacher/gpt2-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run GPT-2 Small (124M) · needs ~0.1 GB · from huggingface.co/mradermacher
chat in the terminal
ollama run hf.co/mradermacher/gpt2-GGUF:Q4_K_M

MLX (Mac)

version

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run GPT-2 Small (124M) · needs ~0.1 GB, Mac only · from huggingface.co/jesusoctavioas
chat in the terminal
mlx_lm.chat --model jesusoctavioas/gpt2-mlx-4Bit
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model jesusoctavioas/gpt2-mlx-4Bit

Other ways to run it

Run it as published · needs ~0.55 GB · from huggingface.co/openai-community
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='openai-community/gpt2')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Weights and documentation: huggingface.co/openai-community ↗

$ how the score was worked out · Surface Form Competition (Hol… 85%, PROST (Aroca-Ouellette et al.… 9%, COPEN — EMNLP 2022 original z… 2% and 2 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runs	7	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 85%	CC BY 4.0
PROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runs	1	▇▇▇ 9%	CC BY 4.0
COPEN — EMNLP 2022 original zero-shot evaluations	3	▇ 2%	CC BY 4.0
EuroEval leaderboards (results collection), English generative leaderboard	2	▇ 1%	MIT
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
prior · pull toward the middle		▇ 3%
weighted average		= -9.5

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json