whatmodel.sh / GPT-2 XL (1.5B)

GPT-2 XL (1.5B)

+ contribute

GPT-2 XL (1.5B) is an open-weights language model from OpenAI, released November 2019. Its published size is 1.558B parameters and a 1k-token context window. Licence: mit.

Catalogue identity notes

OpenAI's released GPT-2 checkpoint as hosted at openai-community/gpt2-xl (formerly the root-namespace 'gpt2-xl' repo). Base (pretrained) model; parameter count as corrected in openai/gpt-2 README.

11.4±1.1#705 of 719 LLMsfrom 29 benchmark results across 8 sources

$ how it compares

The LLMs ranked either side of GPT-2 XL (1.5B), out of 828 with a score, then more from OpenAI.

#	model	score	where	RAM	released
811	Flan-T5 Small Google	14±3	local	~1.8GB	2022-10
812	GPT-Neo 1.3B EleutherAI	14±2	local	~7.1GB	2021-03
813	GPT-2 XL (1.5B) OpenAI	11±1	local	~2.8GB	2019-11
814	BLOOM 1.1B BigScience	11±1	local	~2.6GB	2022-07
815	Pythia 410M EleutherAI	10±2	local	~1.8GB	2023-02
	more from OpenAI
1	GPT-6 Astra OpenAI	87±1	api $20.00/M		2026-09
3	GPT-6.1 Sol OpenAI	85±1	api $4.00/M		2026-09
7	GPT-6 Sol OpenAI	83±1	api $4.00/M		2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

math█···█······█ #2 of 3 * · 2.4% of its score · 1 test

Beats GPT-Neo 1.3B here, though they rank higher overall.

test	result	vs the others in its run	difficulty	counts	why
MathQA	23.6% ✎	█···█··█···█ beats 0/3	hard · avg 46%	▇ 2.4%	too hard for it

other tests20 not in a task · 97% of its score

test	result	vs the others in its run	difficulty	counts	why
HellaSwag +2	40.0% ✎	▃▃▃··▅····▃█ beats 0/8	very easy · avg 92%	▇▇▇▇▇▇▇ 17%	at its level · several runs
ARC Challenge +1	25.0% ✎	▅····▃▃···▅█ beats 0/8	hard · avg 46%	▇▇▇▇▇ 13%	sharp · too hard for it · several runs
OpenBookQA +1	22.4% ✎	█······▂·▆·▂ beats 0/8	very hard · avg 26%	▇▇▇▇▇ 12%	too hard for it · several runs
ARC Easy +1	58.3% ✎	▅·▃▃··▃··▃·█ beats 1/8	very easy · avg 90%	▇▇▇ 8.4%	at its level · several runs
WinoGrande +1	58.3% ✎	█··█···█▄▄·▄ beats 1/8	easy · avg 90%	▇▇▇ 7.9%	several runs
PROST physical reasoning (zero-shot, 10-concept macro average)	30.1% ✎	█···██·█···█ beats 4/4	easy · avg 79%	▇▇▇ 7.4%	too hard for it
PIQA +1	70.5% ✎	▄▄··▄·▄·▄·██ beats 1/8	easy · avg 89%	▇▇ 5.7%	blunt · at its level · several runs
BoolQ	61.8% ✎	▄█▄···▄··█▄▄ beats 1/8	easy · avg 92%	▇▇ 4.6%	typical

12 more tests

test	result	vs the others in its run	difficulty	counts	why
LAMBADA (OpenAI version), last-word accuracy	51.2% ✎	█·▄··█··█··▄ beats 0/7	very easy · avg 90%	▇▇ 4.6%	blunt · at its level
CommonsenseQA	38.6% ✎	█·····█·█··█ beats 3/3	easy · avg 83%	▇▇ 4.0%	at its level
COPA	69.0% ✎	▄·▄·█··▆▂··▂ beats 6/12	very easy · avg 96%	▇ 2.8%	at its level
Story Cloze Test	67.6% ✎	█····█···█·█ beats 3/3	easy · avg 92%	▇ 2.2%	blunt · at its level
Social IQA	39.4% ✎	█▄····█▄▄··▄ beats 0/7	medium · avg 63%	▇ 1.9%	blunt
PubMedQA +1	60.4% ✎	▂·▂▆·▄▂▄▄▂█▂ beats 4/17	medium · avg 75%	▇ 1.6%	blunt · at its level · several runs
MedMCQA	30.9% ✎	▂▂··▄▄▂▂·▄██ beats 3/17	medium · avg 68%	▇ 1.2%	too hard for it
MedQA (USMLE, 4 options)	28.0% ✎	▁▁··▄▁▁··▄▃█ beats 4/17	easy · avg 76%	▇ 1.1%	sharp · too hard for it
Open PT-LLM Leaderboard average (9 Portuguese tasks)	25.9% ✎	▃▁▁▁▁▁▁▁▂█▅▁ beats 23/276	medium · avg 73%	▇ 0.6%	too hard for it · noisy
COPEN concept identification in context (zero-shot, full prompt)	37.0% ✎	▃···▃▃·▃·▃·█ beats 4/7	hard · avg 44%	▇ 0.5%	blunt
COPEN conceptual similarity judgment (zero-shot, full prompt)	11.7% ✎	▄···█···██·▄ beats 6/7	very hard · avg 11%	▇ 0.0%	blunt · too hard for it
COPEN conceptual property judgment (zero-shot, full prompt)	47.1% ✎	█·······▁··▁ beats 3/7	very hard · avg 51%	▇ 0.0%	blunt · too hard for it · noisy

$ where to run it

llama.cpp · recommended

version

Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
macOS / Linux
curl -LsSf https://llama.app/install.sh | sh
Windows
irm https://llama.app/install.ps1 | iex
other ways to install ↗
Run GPT-2 XL (1.5B) · needs ~1.2 GB · from huggingface.co/mradermacher
chat in the terminal
llama cli -hf mradermacher/gpt2-xl-GGUF:Q4_K_M
Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf mradermacher/gpt2-xl-GGUF:Q4_K_M

LM Studio

version

Install LM Studio · Open the app once before using the lms command.
download it (macOS / Windows / Linux) ↗
Run GPT-2 XL (1.5B) · needs ~1.2 GB · from huggingface.co/mradermacher
open in LM Studio ↗
Or download it in the terminal
lms get https://huggingface.co/mradermacher/gpt2-xl-GGUF@Q4_K_M -y
Then run lms ls to see its name, and lms chat followed by that name

Ollama

version

Install Ollama
download it (macOS / Windows) ↗
Linux
curl -fsSL https://ollama.com/install.sh | sh
Run GPT-2 XL (1.5B) · needs ~1.2 GB · from huggingface.co/mradermacher
chat in the terminal
ollama run hf.co/mradermacher/gpt2-xl-GGUF:Q4_K_M

MLX (Mac)

Install MLX (Mac) · Apple silicon Macs only, text only.
with uv
uv tool install mlx-lm
or
pip install mlx-lm
Run GPT-2 XL (1.5B) · needs ~3.1 GB, Mac only · from huggingface.co/warshanks
chat in the terminal
mlx_lm.chat --model warshanks/gpt2-xl-bf16
Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model warshanks/gpt2-xl-bf16

Other ways to run it

Run it as published · needs ~6.43 GB · from huggingface.co/openai-community
uv run --with torch --with 'transformers<5' python -c "from transformers import pipeline; print(pipeline('text-generation', model='openai-community/gpt2-xl')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Weights and documentation: huggingface.co/openai-community ↗

$ how the score was worked out · Surface Form Competition (Hol… 41%, Textbooks Are All You Need II… 25%, EleutherAI GPT-J-6B model car… 19% and 5 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Surface Form Competition (Holtzman et al., EMNLP 2021) — authors' own GPT-2 runs	7	▇▇▇▇▇▇▇▇▇▇▇▇ 41%	CC BY 4.0
Textbooks Are All You Need II: phi-1.5 technical report — authors' own runs	8	▇▇▇▇▇▇▇▇ 25%	CC BY 4.0
EleutherAI GPT-J-6B model card — EleutherAI lm-evaluation-harness runs	4	▇▇▇▇▇▇ 19%	Apache-2.0
PROST (Aroca-Ouellette et al., Findings of ACL-IJCNLP 2021) - authors' own zero-shot runs	1	▇▇ 7%	CC BY 4.0
EleutherAI GPT-Neo 2.7B model card — EleutherAI evaluation-harness runs	2	▇ 3%	MIT
Open Medical-LLM Leaderboard results (HF dataset openlifescienceai/results)	3	▇ 3%	Apache-2.0
COPEN — EMNLP 2022 original zero-shot evaluations	3	▇ 1%	CC BY 4.0
Open Portuguese LLM Leaderboard (archived)	1	▇ 1%	Apache-2.0
prior · pull toward the middle		▇ 1%
weighted average		= 11.4

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-10-11 02:20 UTC · sources and licences · json