whatmodel.sh / Grok 2 (1212)

Grok 2 (1212)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Grok 2 (1212) is a closed-weights language model from SpaceXAI (xAI), released December 2024. Licence: proprietary.

Catalogue identity notes

Comparison model from the Epoch AI internal-evaluation CSVs (Epoch release date 2024-12-12). Dated API model grok-2-1212. Not listed on xAI's current models page (observed 2026-09-28).

56.1±0.9comparison only: no current run option verifiedfrom 7 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of Grok 2 (1212), out of 828 with a score, then more from SpaceXAI (xAI).

#modelscorewhereRAMreleased
292Sky-T1 32B Preview NovaSky (UC Berkeley)56±5local~21GB2025-01
293Qwen Plus (2025-01-25) Alibaba56±1api $0.60/M2025-01
294Grok 2 (1212) SpaceXAI (xAI)56±1comparison only2024-12
295Qwen3-VL 4B Thinking Alibaba56±1local~4.1GB2025-10
296EXAONE Deep 32B LG AI Research56±2local~22GB2025-03
more from SpaceXAI (xAI)
30Grok 4.6 SpaceXAI (xAI)77±1api $3.00/M2026-08
42Grok 4.5 SpaceXAI (xAI)75±1api $3.00/M2026-07
47Grok 4.7 SpaceXAI (xAI)75±1api $3.00/M2026-09

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▅▇█▇▅▄▃▁ #226 of 524 * · 15% of its score · 1 test

Beats Gemini 2.5 Flash, Qwen3.5 4B, Qwen3 30B-A3B Instruct 2507 here, though they rank higher overall. Trails QwQ-32B-Preview, Ministral 3 3B Reasoning 2512, DeepSeek-Coder-V2 Instruct (0724) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
CRUXEval (ZeroEval, 800 problems)75.3% ✎▁▂▆▃▆█▆▃▂·▁▁ beats 38/47medium · avg 61%▇▇▇▇▇▇ 15%sharp
math▁▂▃▅▇▅▅█▄▁▁▁ #253 of 462 · 33% of its score · 3 tests

Beats GPT-4o (2024-08-06), Command A (03-2025), GPT-4o (2024-11-20) here, though they rank higher overall. Trails Olmo 3 7B Instruct, DeepSeek R1 Distill Llama 8B, Phi-4-mini-reasoning 3.8B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
MATH level 563.5% ✎▂▂▅▆▄█▆▃▅▂▃▄ beats 54/92hard · avg 32%▇▇▇▇▇▇▇ 17%sharp · at its level
MATH Level 5 (ZeroEval, 721 problems)60.9% ✎▁▁▁▆█▂▇▇▄··▂ beats 35/38hard · avg 34%▇▇▇▇ 11%sharp · at its level
OTIS mock AIME 2024–202511.5% ✎▃▃▄▃▃▃▇▇█▃▂▅ beats 51/193very hard · avg 8.5%▇▇ 5.5%sharp · at its level · noisy
other tests3 not in a task · 52% of its score
testresultvs the others in its rundifficultycountswhy
MMLU-Redux (ZeroEval, 2,778 questions)87.4% ✎▁·▁▁▄▄▄▅█▄▂▁ beats 49/53easy · avg 77%▇▇▇▇▇▇▇▇▇ 23%typical
ZebraLogic (ZeroEval, 1,000 puzzles)27.7% ✎▁▁▃▆██▁▁·▁▁▁ beats 44/57hard · avg 18%▇▇▇▇▇▇▇ 16%sharp · at its level
GPQA Diamond53.8% ✎▁▁▁▁▂▄▆▄▅▇█▅ beats 75/208hard · avg 40%▇▇▇▇▇ 12%sharp · at its level

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · WildEval ZeroEval published r… 65%, Epoch AI — internal evaluatio… 34%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
WildEval ZeroEval published results4▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 65%Apache-2.0
Epoch AI — internal evaluations3▇▇▇▇▇▇▇▇▇▇ 34%CC BY 4.0
prior · pull toward the middle▇ 0%
weighted average= 56.1

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json