whatmodel.sh / gpt-oss-20b

gpt-oss-20b

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

gpt-oss-20b is an open-weights language model from OpenAI, released August 2025. Its published size is 21B parameters (3.6B active per token) and a 131k-token context window. Licence: apache-2.0.

58.4±0.6#213 of 719 LLMsfrom 64 benchmark results across 21 sources

$ how it compares

The LLMs ranked either side of gpt-oss-20b, out of 828 with a score, then more from OpenAI.

#modelscorewhereRAMreleased
255Mercury 2 Inception59±2comparison only
256DiffusionGemma 26B-A4B Google59±1comparison only2026-06
257gpt-oss-20b OpenAI58±1api $0.13/M +local~14GB2025-08
258Llama 3.3 Nemotron Super 49B v1 NVIDIA58±2local~22GB2025-03
259Olmo 3.1 32B Think Ai258±1local~21GB2025-12
more from OpenAI
1GPT-6 Astra OpenAI87±1api $20.00/M2026-09
3GPT-6.1 Sol OpenAI85±1api $4.00/M2026-09
7GPT-6 Sol OpenAI83±1api $4.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #202 of 300 · 11% of its score · 10 tests

Beats o1 (2024-12-17), o3-mini, DeepSeek R1 (Jan '25) here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Mellum2 12B-A2.5B Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
BrowserGym WorkArena-L1 (AgentLab GenericAgent)38.5% ✎█···█▄▄·▄█·█ beats 2/10hard · avg 29%▇ 1.7%at its level
SWE-bench Verified lab60.7% ✎▁▁▁▁▁▁▂▄█▆▁▁ beats 56/131hard · avg 30%▇ 1.6%sharp · at its level · small run
Terminal-Bench 2.0 with Terminus 2 agent (benchmark authors' runs)3.1% ✎▃··▃·▃▃██▅▃▅ beats 0/14very hard · avg 5.8%▇ 1.5%sharp
Terminal-Bench Hard9.0% ✎▂▅▆▂▂▆▂▂█▅▅▂ beats 12/27very hard · avg 4.7%▇ 1.4%sharp · small run
τ-bench Retail lab54.8% ✎▁····▃·▂▇█▂▆ beats 6/28medium · avg 37%▇ 1.3%at its level · small run
BrowserGym WorkArena-L2 (AgentLab GenericAgent)2.6% ✎▃▅▃·▅▃▃··█·▃ beats 2/11very hard · avg 3.9%▇ 1.2%sharp · noisy
BrowserGym MiniWoB (AgentLab GenericAgent)64.0% ✎▅····▅▅▃▃·▃█ beats 3/11medium · avg 55%▇ 1.0%blunt · at its level
τ-bench Airline lab38.0% ✎▁▁▁·▅▄▁█▇▁▁▇ beats 8/28hard · avg 25%▇ 0.9%at its level · small run
2 more tests
testresultvs the others in its rundifficultycountswhy
Humanity's Last Exam with tools lab17.3% ✎▃▂▂▁▄▂▃▆█▇▂▆ beats 2/54very hard · avg 12%▇ 0.7%small run
T1-Bench multi-domain conversational agent pass^38.6% ✎▂▃··▂·▂·▂▂█▄ beats 1/11hard · avg 25%▇ 0.0%blunt · at its level · noisy
coding▁▁▁▃▆██▇▆▄▃▁ #167 of 458 · 12% of its score · 6 tests

Beats Qwen3-Coder-Next (80B-A3B), Qwen3-VL 235B-A22B Instruct, Qwen3-Next 80B-A3B Instruct here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 8B Reasoning 2512, Phi-4-reasoning 14B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Aider polyglot (225 exercises, two attempts) lab34.2% ✎▂▁▁▃▆▃▄▆█▃▄▂ beats 29/71very hard · avg 8.8%▇▇ 3.9%sharp · at its level · small run
Luau Leetcode fully solved (Torpedo Software)52.4% ✎▁▁▁▁·▁▃▃▅▃█▇ beats 10/49hard · avg 18%▇ 2.2%at its level
LiveCodeBench v6 (window unspecified)72.0% ✎█·▄·▄····▄·▄ beats 5/5medium · avg 42%▇ 2.0%at its level
SWE-bench Verified lab60.7% ✎▁▁▁▁▁▁▂▄█▆▁▁ beats 56/131hard · avg 30%▇ 1.6%sharp · at its level · small run
LiveCodeBench v6 (Feb–May 2025)63.8% ✎▁▁▃▁█▅▄▇▅▅▃█ beats 15/38medium · avg 41%▇ 1.4%at its level
SciCode main problems6.2% ✎▄··▄····█▄·▄ beats 2/5very hard · avg 3.8%▇ 0.5%too hard for it
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #198 of 393 * · 3.4% of its score · 2 tests

Beats Granite 4.2 8B, Qwen3-VL 30B-A3B Thinking, Mistral Small 4 (119B-A6.5B) here, though they rank higher overall. Trails LFM2.5 1.2B Instruct, LFM2.5 1.2B Thinking, Gemma 3 12B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1317.6 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 149/379avg 1268▇ 2.8%blunt
IFEval (aggregation unspecified)92.8% ✎▁▁▃▁▂▄▄▆█▃▃▁ beats 129/141easy · avg 80%▇ 0.6%typical
math▁▂▃▅▆▅▄█▄▁▁▁ #137 of 398 · 23% of its score · 5 tests

Beats Claude Sonnet 4.6, MiniMax-M3, MiMo-V2.5 (310B-A15B) here, though they rank higher overall. Trails Olmo 3 32B Think, Olmo 3.1 32B Think here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
AIME 2024 +283.3% ✎█▄···█·····▄ beats 3/5medium · avg 62%▇▇▇ 8.1%sharp · several runs
AIME 2025 +284.4% ✎▄·█··▄····▄▄ beats 5/5medium · avg 53%▇▇▇ 7.3%sharp · several runs
HMMT 2025 (edition unspecified)64.8% ✎▄··▄▄·▄····█ beats 4/5hard · avg 29%▇ 3.6%sharp · at its level
OTIS mock AIME 2024–202550.8% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 77/193very hard · avg 8.5%▇ 2.6%sharp · at its level · noisy
MATH-50094.8% ✎▄·▄·····█··█ beats 0/5very easy · avg 92%▇ 1.5%maxed out
web search█··········█ #2 of 2 * · 0.7% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
Humanity's Last Exam with tools lab17.3% ✎▃▂▂▁▄▂▃▆█▇▂▆ beats 2/54very hard · avg 12%▇ 0.7%small run
other tests29 not in a task · 52% of its score
testresultvs the others in its rundifficultycountswhy
GPQA Diamond +346.0% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 52/208hard · avg 40%▇▇▇▇▇▇ 15%sharp · at its level · several runs
Humanity's Last Exam +29.8% ✎█··██···█·██ beats 4/5very hard · avg 3.8%▇▇ 4.4%several runs
Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations)0.5198 ✎▁▄▃▃▇▄█▃▅▁▁▁ beats 21/30avg 0.423▇▇ 4.3%sharp
MMLU-Pro +175.6% ✎█··█··███··█ beats 4/5medium · avg 57%▇ 3.6%at its level · several runs
MMLU +181.7% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 158/183medium · avg 72%▇ 3.5%several runs
MMMLU +177.4% ✎▂▁▁▂▃▂▄▄▅██▃ beats 25/46medium · avg 64%▇ 2.4%at its level · several runs
Medmarks-V weighted mean win rate (61 models, 71 configurations)53.6% ✎▂▃▅▂▆▃█▅█▅▃▆ beats 21/34medium · avg 42%▇ 2.1%blunt · at its level
EQ-Bench 3 (emotional intelligence, judge Elo)1111.5 ✎▁·▂▁▄▅▅▃█▄▁▁ beats 14/62avg 1044▇ 1.7%typical
21 more tests
testresultvs the others in its rundifficultycountswhy
SimpleQA +16.7% ✎▄▄▄▆█▆███·▂█ beats 7/32very hard · avg 6.4%▇ 1.6%small run · several runs
BRIDGE clinical text, zero-shot average (87 tasks)29.0% ✎▁▁▁▁▁▂▂▃▄▆█▃ beats 28/97hard · avg 32%▇ 1.5%blunt · at its level
AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge)30.6% ✎▁▁·▁▂▂▄█▄▅▄▄ beats 27/66hard · avg 31%▇ 1.5%at its level
RobloxQA v2.0 multiple choice (Torpedo Software)88.7% ✎▁▁▃▃█▄▃▃▆▂▂▂ beats 8/49easy · avg 78%▇ 1.4%typical
Open Persian LLM Leaderboard v2 average (21 tasks)45.5% ✎▂▄▂▃█▄▃▆▃▅▂▂ beats 19/81medium · avg 58%▇ 1.1%blunt · at its level
SuperGPQA50.1% ✎▂▁▂▂▅▆▃▇▇▇█▃ beats 19/46medium · avg 37%▇ 1.1%at its level
KOR-Bench72.3% ✎▄·······█··▄ beats 3/3medium · avg 55%▇ 0.8%at its level
LongSocialBench (full-context, 4-option MCQ over discussion threads)36.3% ✎▆▂·▄█·▂▂·▄▄▂ beats 1/16very hard · avg 36%▇ 0.8%blunt
MMLU-ProX69.1% ✎▁·▁▁▁▂▃▃▂█▅▄ beats 7/35medium · avg 56%▇ 0.7%at its level · small run
SurgiQ surgical MCQ (zero-shot, log-probability scoring)57.6% ✎▂···▂▂·▃▂▃██ beats 9/17medium · avg 53%▇ 0.7%at its level
RULER 128k78.7% ✎▃··▅▄▇▅█▇·▃▃ beats 23/32easy · avg 72%▇ 0.6%typical
IFBench (aggregation unspecified)69.4% ✎███···██···█ beats 5/5hard · avg 35%▇ 0.6%at its level
Open PT-LLM Leaderboard average (9 Portuguese tasks)42.8% ✎▃▁▁▁▁▁▃▁▂█▅▁ beats 51/276medium · avg 73%▇ 0.5%at its level · noisy
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy67.7% ✎▃·▁▁▂▁▂▅█▇▄▁ beats 44/100medium · avg 68%▇ 0.4%noisy
Chess Puzzles (Epoch)0.0% ✎█▁▁▁▁▂▁▄▂▂▁▁ beats 19/135very hard · avg 5.9%▇ 0.4%sharp · too hard for it · noisy
BeyondAIME69.0% ✎▄·▂·▂▄··▄▄▄█ beats 12/15medium · avg 36%▇ 0.3%at its level
HealthBench lab42.5% ✎▃▂·▂·▅█▃▂·▃▃ beats 0/17hard · avg 31%▇ 0.3%blunt · at its level · small run
HealthBench Hard lab10.8% ✎▃····▁▁▂█·▁▁ beats 0/13very hard · avg 13%▇ 0.2%blunt · small run
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy72.7% ✎▁▁▁▁▁▁▁▂▄█▄▃ beats 42/146easy · avg 83%▇ 0.2%noisy
Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate)96.3% ✎▁▁▂▃▄▄█▆▃▂▂▁ beats 95/140very easy · avg 94%▇ 0.1%blunt · maxed out · noisy
miniF2F (pass@1)12.0% ✎▃·█▅·····▅·▃ beats 3/8very hard · avg 14%▇ 0.1%blunt · noisy · small run

$ where to run it

Through an API

api · Groq · $0.13/M · $0.075 in / $0.3 out per 1M tokens · 131k context · console.groq.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.

llama.cpp · recommended

  1. Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.
    macOS / Linux
    curl -LsSf https://llama.app/install.sh | sh
    Windows
    irm https://llama.app/install.ps1 | iex

    other ways to install ↗

  2. Run gpt-oss-20b · needs ~11.6 GB · from huggingface.co/unsloth
    chat in the terminal
    llama cli -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M
  3. Or start a local server and open the address it prints (http://localhost:9931)
    llama serve -hf unsloth/gpt-oss-20b-GGUF:Q4_K_M

LM Studio

version

  1. Install LM Studio · Open the app once before using the lms command.

    download it (macOS / Windows / Linux) ↗

  2. Run gpt-oss-20b · needs ~11.6 GB · from huggingface.co/unsloth

    open in LM Studio ↗

  3. Or download it in the terminal
    lms get https://huggingface.co/unsloth/gpt-oss-20b-GGUF@Q4_K_M -y
  4. Then run lms ls to see its name, and lms chat followed by that name

Ollama

  1. Install Ollama

    download it (macOS / Windows) ↗

    Linux
    curl -fsSL https://ollama.com/install.sh | sh
  2. Run gpt-oss-20b · needs ~13.1 GB · from ollama.com
    chat in the terminal
    ollama run gpt-oss:20b

MLX (Mac)

version

  1. Install MLX (Mac) · Apple silicon Macs only, text only.
    with uv
    uv tool install mlx-lm
    or
    pip install mlx-lm
  2. Run gpt-oss-20b · needs ~11.8 GB, Mac only · from huggingface.co/InferenceIllusionist
    chat in the terminal
    mlx_lm.chat --model InferenceIllusionist/gpt-oss-20b-MLX-4bit
  3. Or start an OpenAI-compatible server at http://localhost:8080/v1
    mlx_lm.server --model InferenceIllusionist/gpt-oss-20b-MLX-4bit

Official weights and documentation: huggingface.co/openai ↗

$ how the score was worked out · Falcon-H1R: Pushing the Reaso… 23%, ByteDance-Seed/Seed-OSS-36B-I… 20%, OpenAI — gpt-oss-120b & gpt-o… 20% and 18 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling10▇▇▇▇▇▇▇ 23%CC BY 4.0
ByteDance-Seed/Seed-OSS-36B-Instruct model card14▇▇▇▇▇▇ 20%apache-2.0
OpenAI — gpt-oss-120b & gpt-oss-20b Model Card · lab-reported14▇▇▇▇▇▇ 20%CC BY 4.0
Epoch AI — internal evaluations3▇▇ 8%CC BY 4.0
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks — authors' own runs of 61 models2▇▇ 6%CC BY 4.0
ServiceNow BrowserGym leaderboard (AgentLab GenericAgent runs)3▇ 4%MIT
Roblox LLM Leaderboard (Torpedo Software): RobloxQA v2.0 and Luau Leetcode2▇ 4%MIT
LMArena leaderboard dataset1▇ 3%CC BY 4.0
EQ-Bench 3 canonical leaderboard results1▇ 2%MIT
BRIDGE Medical Leaderboard (Harvard Medical School / Brigham and Women's YLab)1▇ 2%MIT
Inception Arabic/Hindi leaderboards (AraGen-v3, HindiGen-v1, Arabic IFEval)1▇ 2%Apache-2.0
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — benchmark authors' own agent-model trials1▇ 1%CC BY 4.0
Multiverse Computing HyperNova 60B 2602 model card: evaluation table (own runs incl. gpt-oss baselines)1▇ 1%Apache-2.0
Open Persian LLM Leaderboard v2 (Part DP AI and AUT NLP Lab)1▇ 1%Apache-2.0
NVIDIA — Nemotron 3 Nano technical report2▇ 1%CC BY 4.0
LongSocialBench: Do Long-Context LLMs Understand Online Discussion Threads? (arXiv:2610.04118v1)1▇ 1%CC BY 4.0
SurgiQ: A Large-Scale Multi-Domain Benchmark for Evaluating Surgical Understanding in Large Language Models (arXiv:2606.08071v1)1▇ 1%CC BY 4.0
EuroEval leaderboards (results collection), English generative leaderboard2▇ 1%MIT
Open Portuguese LLM Leaderboard (archived)1▇ 0%Apache-2.0
Vectara hallucination leaderboard (vectara/hallucination-leaderboard repository)1▇ 0%Apache-2.0
T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domains (arXiv:2606.11070v1)1▇ 0%CC BY 4.0
prior · pull toward the middle▇ 0%
weighted average= 58.4
$ published speed measurements · 11

Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.

NVIDIA DGX Spark (GB10)MXFP483.43 tok/sllama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32; 32 generated tokens; context depth 0; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
NVIDIA DGX Spark (GB10)MXFP479.22 tok/sllama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d4096; 32 generated tokens; context depth 4096; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
NVIDIA DGX Spark (GB10)MXFP461.65 tok/sllama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d32768; 32 generated tokens; context depth 32768; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
Apple M2 Ultra (GPU core count and RAM not specified)MXFP4129.97 tok/sllama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32; 32 generated tokens; context depth 0; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
Apple M2 Ultra (GPU core count and RAM not specified)MXFP4123.38 tok/sllama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d4096; 32 generated tokens; context depth 4096; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
Apple M2 Ultra (GPU core count and RAM not specified)MXFP498.25 tok/sllama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d32768; 32 generated tokens; context depth 32768; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
NVIDIA DGX Spark (GB10)MXFP475.22 tok/sllama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d8192; 32 generated tokens; context depth 8192; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
NVIDIA DGX Spark (GB10)MXFP470.36 tok/sllama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d16384; 32 generated tokens; context depth 16384; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
Apple M2 Ultra (GPU core count and RAM not specified)MXFP4117.39 tok/sllama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d8192; 32 generated tokens; context depth 8192; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
Apple M2 Ultra (GPU core count and RAM not specified)MXFP4109.75 tok/sllama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d16384; 32 generated tokens; context depth 16384; n_ubatch=2048; flash attention=1; one sequence

Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred.

source ↗
AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S; pre-production Framework Desktop 128GB LPDDR5x-8000MXFP4 (summary-table label 'OpenAI gpt-oss 20B MXFP4'; result folder/file named gpt-oss-20b-F16)47.36 tok/sllama.cpp 6c7e9a544 (commit from system_info.json), backend hip · llama-bench tg128 (128 generated tokens, empty context); backend variant 'llama.cpp-hip'; flags: defaults; mmap disabled per the guide; value = bold best tg128 in this model's Benchmark Results table

Author's own sweep; README value 47.357 rounded to 2 decimals. Kernel 6.16.0-1-mainline; system_info timestamp 2025-08-08T14:48:13.390941Z; CPU string 'AMD Eng Sample'. Pre-production hardware and BIOS per the README.

source ↗

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json