whatmodel.sh / gpt-oss-120b
gpt-oss-120b
↗ share
+ contribute
gpt-oss-120b is an open-weights language model from OpenAI, released August 2025. Its published size is 117B parameters (5.1B active per token) and a 131k-token context window. Licence: apache-2.0.
$ how it compares
The LLMs ranked either side of gpt-oss-120b, out of 828 with a score, then more from OpenAI.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 207 |  Gemini 2.5 Flash Preview (05-20) | 62±1 | comparison only | ||
| 208 |  A.X K1 (519B-A33B) | 62±1 | local | ||
| 209 | gpt-oss-120b | 62±1 | api $0.26/M +local | ||
| 210 | GPT-5 nano | 62±1 | api $0.14/M | ||
| 211 |  Qwen3-VL 32B Thinking | 62±1 | api $0.28/M +local | ||
| more from OpenAI | |||||
| 1 | GPT-6 Astra | 87±1 | api $20.00/M | ||
| 3 | GPT-6.1 Sol | 85±1 | api $4.00/M | ||
| 7 | GPT-6 Sol | 83±1 | api $4.00/M | ||
$ what it's good at
agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #146 of 300 · 18% of its score · 13 tests
Beats o4-mini, o1 (2024-12-17), o3-mini here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| SWE-bench Verified +1 | 31.0% ✎ | █·█▄·······█ beats 3/6 | 3.0% |
| Terminal-Bench 2.0 with Terminus 2 agent (benchmark authors' runs) | 18.7% ✎ | ▃··▃·▃▃██▅▃▅ beats 3/14 | 1.9% |
| τ²-bench Retail | 61.4% ✎ | █▄··▄··▄·▄·▄ beats 1/6 | 1.8% |
| BrowserGym WorkArena-L1 (AgentLab GenericAgent) | 50.9% ✎ | █···█▄▄·▄█·█ beats 6/10 | 1.7% |
| BrowserGym WorkArena-L2 (AgentLab GenericAgent) | 11.5% ✎ | ▃▅▃·▅▃▃··█·▃ beats 6/11 | 1.6% |
| Terminal-Bench Hard | 22.0% ✎ | ▂▅▆▂▂▆▂▂█▅▅▂ beats 16/27 | 1.5% |
| τ²-bench Airline | 45.3% ✎ | ▄▄······▄█▄▄ beats 0/6 | 1.3% |
| τ²-bench Telecom | 57.7% ✎ | ▁·▅█▆▅▂▃▃▂▃▁ beats 49/89 | 1.3% |
5 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| τ-bench Retail lab | 67.8% ✎ | ▁····▃·▂▇█▂▆ beats 16/28 | 1.3% |
| τ-bench Airline lab | 49.2% ✎ | ▁▁▁·▅▄▁█▇▁▁▇ beats 21/28 | 0.9% |
| BrowserGym MiniWoB (AgentLab GenericAgent) | 66.4% ✎ | ▅····▅▅▃▃·▃█ beats 6/11 | 0.9% |
| Humanity's Last Exam with tools lab | 19.0% ✎ | ▂▃▂▁▄▂▃▆█▇▂▆ beats 4/54 | 0.8% |
| T1-Bench multi-domain conversational agent pass^3 | 29.3% ✎ | ▂▂··▂·▂·▂▃█▄ beats 5/11 | 0.0% |
coding▁▁▁▃▆██▇▆▄▃▁ #157 of 458 · 25% of its score · 4 tests
Beats Gemini 2.5 Pro, Qwen3-Coder-Next (80B-A3B), Grok 4.1 Fast (Reasoning) here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 8B Reasoning 2512, Phi-4-reasoning 14B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Aider polyglot (225 exercises, two attempts) +2 | 41.8% ✎ | ▂·▂▃▅▂▃█▆▄▂▂ beats 21/49 | 20% |
| SWE-bench Verified +1 | 31.0% ✎ | █·█▄·······█ beats 3/6 | 3.0% |
| WebApp1K-Duo-React pass@1 (onekq-ai) | 75.0% ✎ | ▁··▁·▁▁▁▆█▇█ beats 39/45 | 1.4% |
| LiveCodeBench v6 (Aug 2024–May 2025) | 69.9% ✎ | ▄▄·▄·····█·▄ beats 4/5 | 1.1% |
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #156 of 393 * · 2.7% of its score · 1 test
Beats MiniMax-M2, MiniCPM5 2B, Granite 4.2 30B here, though they rank higher overall. Trails LFM2.5 1.2B Thinking, LFM2.5 8B-A1B, LFM2.5 2.6B here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| LMArena (text) | 1351.6 ✎ | ▁▁▁▂▃▃▃▆▆▇█▃ beats 196/379 | 2.7% |
math▁▂▃▅▆▅▄█▄▁▁▁ #103 of 398 · 14% of its score · 5 tests
Beats Claude Sonnet 4.6, MiniMax-M3, Nemotron 3 Ultra 550B-A55B here, though they rank higher overall. Trails Gemini 2.5 Flash here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AIME 2024 +1 | 75.8% ✎ | █···███·█··█ beats 0/5 | 5.4% |
| AIME 2025 +1 | 77.7% ✎ | █·█···███··█ beats 1/5 | 4.7% |
| OTIS mock AIME 2024–2025 | 88.9% ✎ | ▃▃▄▃▃▃▇▇█▃▂▅ beats 139/193 | 2.8% |
| MathConstraint full release, no tools (329 instances) | 39.5% ✎ | ▄··▄▄·▄▄█▄·▄ beats 4/8 | 0.5% |
| MATH-500 | 96.0% ✎ | ▄······▄▄▄·█ beats 1/5 | 0.4% |
web search█··········█ #1 of 2 * · 0.8% of its score · 1 test
| test | result | vs the others in its run | counts |
|---|---|---|---|
| Humanity's Last Exam with tools lab | 19.0% ✎ | ▂▃▂▁▄▂▃▆█▇▂▆ beats 4/54 | 0.8% |
other tests29 not in a task · 43% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond +2 | 75.8% ✎ | ▁▁▁▁▂▄▆▄▅▇█▅ beats 115/208 | 13% |
| Medmarks-V mean win rate (19 verifiable medical benchmarks, 71 configurations) | 0.5771 ✎ | ▁▄▃▃▇▄█▃▅▁▁▁ beats 27/30 | 4.2% |
| Humanity's Last Exam +1 | 10.6% ✎ | ▄····▄·█·▄·▄ beats 0/5 | 3.7% |
| Medmarks-V weighted mean win rate (61 models, 71 configurations) | 58.7% ✎ | ▂▃▅▂▆▃█▅█▅▃▆ beats 28/34 | 2.1% |
| KCL-Essay vanilla | 31.6% ✎ | ▁···▁▃▂█▂▃▃▂ beats 10/23 | 2.0% |
| EQ-Bench 3 (emotional intelligence, judge Elo) | 1167.5 ✎ | ▁·▂▁▄▅▅▃█▄▁▁ beats 24/62 | 1.7% |
| MMLU-Pro | 67.1% ✎ | ▁▁·▁▁▁▃▂▄▅█▁ beats 80/188 | 1.7% |
| BRIDGE clinical text, zero-shot average (87 tasks) | 37.2% ✎ | ▁▁▁▁▁▂▂▃▄▆█▃ beats 55/97 | 1.5% |
21 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AraGen-v3 3C3H score (Inception, Claude Sonnet 4 judge) | 43.2% ✎ | ▁▁·▁▂▂▄█▄▅▄▄ beats 37/66 | 1.5% |
| CPTUB average (SpeakLeash, 0-5 judge scale) | 76.4% ✎ | ▁▂▂▂▃█▃▆▅▄▂▃ beats 73/95 | 1.3% |
| MMLU lab | 90.0% ✎ | ▁▁▁▁▁▂▃▅█▃▆▃ beats 178/183 | 1.2% |
| SimpleQA lab | 16.8% ✎ | ▄▄▄▆█▆███·▂█ beats 17/32 | 1.0% |
| KCL-MCQA vanilla | 29.2% ✎ | ▂···█▂▆▆██▆▂ beats 3/23 | 1.0% |
| MMMLU lab | 81.3% ✎ | ▂▁▁▂▃▂▄▄▅██▃ beats 31/46 | 0.9% |
| LongSocialBench (full-context, 4-option MCQ over discussion threads) | 39.5% ✎ | ▆▂·▄█·▂▂·▄▄▂ beats 4/16 | 0.8% |
| Multi-IF (aggregation unspecified) | 83.0% ✎ | ▄·····▄█▄▄·▄ beats 2/6 | 0.8% |
| SurgiQ surgical MCQ (zero-shot, log-probability scoring) | 64.6% ✎ | ▂···▂▂·▃▂▃██ beats 14/17 | 0.7% |
| Chess Puzzles (Epoch) | 20.0% ✎ | █▁▁▁▁▂▁▄▂▂▁▁ beats 81/135 | 0.6% |
| WMT25 Terminology Translation (term success rate) | 83.8% ✎ | ▂·▂··▂·▄▂·█▆ beats 3/12 | 0.6% |
| Mystery Game Puzzles (Epoch) | 0.0% ✎ | █··▂▃▇▄█▂▃▁▁ beats 7/74 | 0.4% |
| Open PT-LLM Leaderboard average (9 Portuguese tasks) | 43.0% ✎ | ▃▁▁▁▁▁▃▁▂█▅▁ beats 52/276 | 0.4% |
| SOOHAK-Mini (702 problems, Avg@3) | 61.0% ✎ | ▄··▄·▄·▄▄█·▄ beats 3/7 | 0.4% |
| SOOHAK Challenge (340 problems, Avg@3) | 11.3% ✎ | █·█████··█·█ beats 4/7 | 0.3% |
| HealthBench lab | 57.6% ✎ | ▂▂·▂·▅█▃▂·▃▃ beats 8/17 | 0.3% |
| HealthBench Hard lab | 30.0% ✎ | ▁····▁▁▃█·▁▁ beats 3/13 | 0.2% |
| WMT24++ | 76.6% ✎ | █·▃·▅▅▃···▃▃ beats 0/10 | 0.2% |
| WMT26 General MT (GEMBA-ESA, LLM judge) | 72.3% ✎ | ▄·▄▄▄·▄██▄█▄ beats 3/12 | 0.2% |
| Vectara hallucination leaderboard, 2025 dataset (HHEM-2.3, factual consistency rate) | 97.6% ✎ | ▁▁▂▃▄▄█▆▃▂▂▁ beats 117/140 | 0.1% |
| Vectara hallucination leaderboard (HHEM-2.3, factual consistency rate) | 85.8% ✎ | ▁▂▁▃▅█▂▃▄▃▁▁ beats 13/98 | 0.0% |
$ where to run it
Through an API
api · Groq · $0.26/M · $0.15 in / $0.6 out per 1M tokens · 131k context · console.groq.com ↗ · price sourceStandard base token rate from models.dev. Provider-specific tiers, long-context surcharges, caching and promotions may differ.
llama.cpp · recommended
- Install llama.cpp · Homebrew and winget installs name the commands llama-cli and llama-server instead.macOS / Linux
curl -LsSf https://llama.app/install.sh | shWindowsirm https://llama.app/install.ps1 | iex - Run gpt-oss-120b · needs ~62.8 GB · from huggingface.co/unslothchat in the terminal
llama cli -hf unsloth/gpt-oss-120b-GGUF:Q4_K_M - Or start a local server and open the address it prints (http://localhost:9931)
llama serve -hf unsloth/gpt-oss-120b-GGUF:Q4_K_M
LM Studio
version
- Install LM Studio · Open the app once before using the lms command.
- Run gpt-oss-120b · needs ~62.8 GB · from huggingface.co/unsloth
- Or download it in the terminal
lms get https://huggingface.co/unsloth/gpt-oss-120b-GGUF@Q4_K_M -y - Then run lms ls to see its name, and lms chat followed by that name
Ollama
- Install Ollama
download it (macOS / Windows) ↗
Linuxcurl -fsSL https://ollama.com/install.sh | sh - Run gpt-oss-120b · needs ~70.7 GB · from ollama.comchat in the terminal
ollama run gpt-oss:120b
MLX (Mac)
version
- Install MLX (Mac) · Apple silicon Macs only, text only.with uv
uv tool install mlx-lmorpip install mlx-lm - Run gpt-oss-120b · needs ~65.8 GB, Mac only · from huggingface.co/egoriowchat in the terminal
mlx_lm.chat --model egoriow/gpt-oss-120b-mlx - Or start an OpenAI-compatible server at http://localhost:8080/v1
mlx_lm.server --model egoriow/gpt-oss-120b-mlx
Official weights and documentation: huggingface.co/openai ↗
$ how the score was worked out · OpenAI — gpt-oss-120b & gpt-o… 19%, Prime Intellect INTELLECT-3 m… 18%, ServiceNow-AI/Apriel-1.6-15b-… 13% and 22 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
$ published speed measurements · 14
Reference results for the hardware, engine and settings below. They aren't verified speeds of the downloads above or predictions for your machine.
| NVIDIA DGX Spark (GB10) | MXFP4 | 58.72 tok/s | llama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32; 32 generated tokens; context depth 0; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| NVIDIA DGX Spark (GB10) | MXFP4 | 55.67 tok/s | llama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d4096; 32 generated tokens; context depth 4096; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| NVIDIA DGX Spark (GB10) | MXFP4 | 42.76 tok/s | llama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d32768; 32 generated tokens; context depth 32768; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| Apple M2 Ultra (GPU core count and RAM not specified) | MXFP4 | 85.6 tok/s | llama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32; 32 generated tokens; context depth 0; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| Apple M2 Ultra (GPU core count and RAM not specified) | MXFP4 | 82.03 tok/s | llama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d4096; 32 generated tokens; context depth 4096; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| Apple M2 Ultra (GPU core count and RAM not specified) | MXFP4 | 65.52 tok/s | llama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d32768; 32 generated tokens; context depth 32768; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| NVIDIA DGX Spark (GB10) | MXFP4 | 52.87 tok/s | llama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d8192; 32 generated tokens; context depth 8192; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| NVIDIA DGX Spark (GB10) | MXFP4 | 49.45 tok/s | llama.cpp 11fb327bf (7941), CUDA · llama-bench; tg32 @ d16384; 32 generated tokens; context depth 16384; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| Apple M2 Ultra (GPU core count and RAM not specified) | MXFP4 | 78.23 tok/s | llama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d8192; 32 generated tokens; context depth 8192; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| Apple M2 Ultra (GPU core count and RAM not specified) | MXFP4 | 73.2 tok/s | llama.cpp b828e18c7 (7948), Metal/BLAS · llama-bench; tg32 @ d16384; 32 generated tokens; context depth 16384; n_ubatch=2048; flash attention=1; one sequence Published token-generation mean; not prompt processing or aggregate batch throughput. Hardware core count and installed RAM are not inferred. | source ↗ |
| AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB | mxfp4 | 55.57 tok/s | llama.cpp 2496f9c14 (build_number 9049 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r20 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 58 (2026-05-07, measured-local): Clean paused-system long-context rerun; pp2048 728.60; pp8192 678.59; pp16384 605.21; pp32768 478.25; pp65536 293.73 r1. | source ↗ |
| AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB | mxfp4 | 50.59 tok/s | llama.cpp 2496f9c14 (build_number 9049 as recorded), Vulkan (RADV) · llama-bench; tg128; 128 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r3 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. Curated in the guide's data/benchmarks.csv line 63 (2026-05-07, measured-local): Split GGUF loaded locally; pp2048 was 707.29 t/s and tg32 was 51.02 t/s; benchmark guarded to preserve workstation health. | source ↗ |
| AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S iGPU; Beelink GTR9 Pro 128GB | mxfp4 | 51.02 tok/s | llama.cpp 2496f9c14 (build_number 9049 as recorded), Vulkan (RADV) · llama-bench; tg32; 32 generated tokens; context depth 0; n_prompt=0 row; n_batch=2048; n_ubatch=512; flash_attn=1; type_k=f16; type_v=f16; n_gpu_layers=999; threads=16; one sequence; file label r3 repetitions Author's own llama-bench CSV row (avg_ts mean over repetitions, stddev_ts retained). GPU string as recorded: Radeon 8060S Graphics (RADV STRIX_HALO). Unified memory; the guide reports ~124GiB OS-visible. Kernel/Mesa/power profile per the run folder notes. Not prompt processing or multi-request throughput. The guide's curated benchmarks.csv note for this run states 'tg32 was 51.02 t/s'. | source ↗ |
| AMD Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S; pre-production Framework Desktop 128GB LPDDR5x-8000 | MXFP4 (summary-table label 'OpenAI gpt-oss 120B MXFP4'; result folder/file named gpt-oss-120b-F16) | 33.66 tok/s | llama.cpp 6c7e9a544 (commit from system_info.json), backend hip · llama-bench tg128 (128 generated tokens, empty context); backend variant 'llama.cpp-hip'; flags: hipBLASLt=1; mmap disabled per the guide; value = bold best tg128 in this model's Benchmark Results table Author's own sweep; README value 33.66161 rounded to 2 decimals. Kernel 6.16.0-1-mainline; system_info timestamp 2025-08-08T14:49:04.072682Z; CPU string 'AMD Eng Sample'. Pre-production hardware and BIOS per the README. | source ↗ |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json