whatmodel.sh / ERNIE 4.5 21B-A3B

ERNIE 4.5 21B-A3B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

ERNIE 4.5 21B-A3B is an open-weights language model from Baidu, released June 2025. Its published size is 21B parameters (3B active per token). Licence: apache-2.0.

Catalogue identity notes

Comparison-only, proposed so the card-ernie-45-posttrained-final runs count. The pinned ERNIE README (@790a50b) prints a post-trained chart with columns 'ERNIE-4.5-0.3B', 'Qwen3-30B-A3B' (competitor, unused) and 'ERNIE-4.5-21B-A3B'; its model table lists the post-trained ERNIE-4.5-21B-A3B release. Weights: https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-PT (sha 87db95487941cb39592ee0abca3b9155a6d19c5c, created 2025-06-28, apache-2.0, 21,948,655,808 params in safetensors metadata; '-PT' = PyTorch weights, as for the catalogued 300B-A47B).

52.0±1.2comparison only: no current run option verifiedfrom 21 benchmark results across 3 sources

$ how it compares

The LLMs ranked either side of ERNIE 4.5 21B-A3B, out of 828 with a score, then more from Baidu.

#modelscorewhereRAMreleased
376GPT-4 (0314) OpenAI52±1comparison only2023-03
377GLM-4 (0520) Z.ai52±1comparison only2024-05
378ERNIE 4.5 21B-A3B Baidu52±1comparison only2025-06
379Bielik 11B v2.2 Instruct SpeakLeash52±3local~8.5GB2024-08
380Granite 4.0 H Small (32B-A9B) IBM52±1api $0.11/M +local~21GB2025-09
more from Baidu
235ERNIE 4.5 300B-A47B Baidu60±1local~97GB2025-06

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▂▆█▆▅▄▄▃▂▁ #311 of 340 * · 3.1% of its score · 1 test

Beats Magistral Small 1.0 (2506) here, though they rank higher overall. Trails Granite 4.0 H 350M, Qwen3 0.6B, Qwen2.5 1.5B Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
EEGAgentBench overall (tool-based EEG analysis agents, 6 tasks)22.8% ✎▃·▁▁··▇▇█▂·▁ beats 0/25hard · avg 29%▇ 3.1%blunt · at its level
coding█··········█ #2 of 2 · 15% of its score · 3 tests
testresultvs the others in its rundifficultycountswhy
HumanEval+ lab89.6% ✎▁·▁▁▂▃▄▄█▅▄▃ beats 131/145easy · avg 83%▇▇▇ 7.4%sharp · small run
LiveCodeBench (version unspecified) lab26.4% ✎▁···▁▁▃▅▅█▄▅ beats 4/37hard · avg 28%▇▇ 4.4%at its level · small run
MBPP+ lab77.5% ✎▁··▁▁▃▄█▆▄▁▁ beats 104/120easy · avg 70%▇ 3.1%at its level · small run
chat & writing█··········█ #2 of 2 * · 2.8% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
IFEval (aggregation unspecified) lab80.0% ✎▁▁▃▁▂▄▄▆█▃▂▁ beats 53/141easy · avg 78%▇ 2.8%small run
math█··········█ #2 of 2 · 30% of its score · 4 tests
testresultvs the others in its rundifficultycountswhy
AIME 2024 lab27.9% ✎▁▁▁▂▁▃▁▅███▃ beats 15/85hard · avg 31%▇▇▇ 8.5%sharp · at its level · small run
GSM8K lab93.3% ✎▂▂▂▅▅▆▅▇█▇▄▃ beats 133/146very easy · avg 90%▇▇▇ 8.2%sharp · small run
AIME 2025 lab19.7% ✎▁·▁▁▁▃▅▆█▇▅▁ beats 10/133hard · avg 21%▇▇▇ 8.1%sharp · at its level · small run
MATH-500 lab87.2% ✎▁▃▂▃▅▇▅▇█▅▂▁ beats 31/84very easy · avg 87%▇▇ 5.2%small run
other tests12 not in a task · 49% of its score
testresultvs the others in its rundifficultycountswhy
BIG-Bench Hard lab83.0% ✎▁▁▁▁▂▄▅▅█▃▄▁ beats 115/129easy · avg 79%▇▇▇ 8.7%sharp · small run
MMLU lab76.0% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 126/183easy · avg 74%▇▇▇ 6.5%at its level · small run
C-Eval lab85.4% ✎▁···▁▁▂▁▂▂▃█ beats 15/33easy · avg 79%▇▇ 5.5%small run
MMLU-Pro lab66.0% ✎▁▁·▁▁▁▃▂▄▅█▁ beats 76/188medium · avg 60%▇▇ 5.3%at its level · small run
CMMLU lab84.8% ✎▅·▅····█▃▅█▅ beats 9/14easy · avg 79%▇▇ 4.4%at its level · small run
SimpleQA lab24.2% ✎▄▄▄▆█▆███·▂█ beats 23/32hard · avg 16%▇▇ 4.2%small run
ChineseSimpleQA lab48.9% ✎▄··▄▄▄··▄··█ beats 0/6medium · avg 52%▇ 3.3%at its level · small run
SysBench (aggregation unspecified) lab48.0% ✎█··█····█··█ beats 0/3medium · avg 52%▇ 3.0%at its level · small run
4 more tests
testresultvs the others in its rundifficultycountswhy
Multi-IF (aggregation unspecified) lab61.0% ✎▁▁▁▁▁▃▃▄█▄▁▂ beats 6/41medium · avg 62%▇ 2.5%blunt · at its level · small run
EuroEval HellaSwag (English), test split, 5-shot generative MC, accuracy83.0% ✎▃·▁▁▂▁▂▅█▇▄▁ beats 79/100medium · avg 68%▇ 2.1%at its level · noisy
MuSR lab67.7% ✎█·██······██ beats 3/4medium · avg 58%▇ 1.8%blunt · at its level · small run
EuroEval Life in the UK (English knowledge MC), test split, 5-shot, accuracy83.7% ✎▁▁▁▁▁▁▁▂▄█▄▃ beats 90/146easy · avg 83%▇ 1.3%noisy

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · ERNIE 4.5 post-trained 300B-A… 93%, EuroEval leaderboards (result… 3%, EEGAgentBench: Benchmarking L… 3%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
ERNIE 4.5 post-trained 300B-A47B benchmark chart · lab-reported18▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 93%Apache-2.0
EuroEval leaderboards (results collection), English generative leaderboard2▇ 3%MIT
EEGAgentBench: Benchmarking LLM Agents on Short- and Long-Horizon EEG Analysis (arXiv:2609.31632v1)1▇ 3%CC BY 4.0
prior · pull toward the middle▇ 1%
weighted average= 52.0

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json