whatmodel.sh / Nova Premier
Nova Premier
↗ share
+ contribute
Nova Premier is a closed-weights language model from Amazon, released April 2025.
$ how it compares
The LLMs ranked either side of Nova Premier, out of 828 with a score, then more from Amazon.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 337 |  Pixtral Large (2411) | 54±3 | local | ||
| 338 |  Bielik 11B v2.6 Instruct | 54±3 | local | ||
| 339 | Nova Premier | 54±2 | api $5.00/M | ||
| 340 |  Mellum2 12B-A2.5B Thinking | 54±1 | local | ||
| 341 | Mistral Large 2 (2407) | 54±1 | local | ||
| more from Amazon | |||||
| 184 | Nova 2 Pro (Preview) | 64±2 | api $3.78/M | ||
| 269 | Nova 2 Lite | 58±2 | api $0.94/M | ||
| 423 | Nova Pro | 50±1 | api $1.40/M | ||
$ what it's good at
agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #157 of 300 * · 25% of its score · 2 tests
Beats o1 (2024-12-17), o3-mini, DeepSeek R1 0528 here, though they rank higher overall. Trails SmolLM3 3B, Jamba 1.5 Mini, Jamba 1.5 Large here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| MASSIVE-Agents AST (10k, zero-shot) lab | 34.0% ✎ | █····███·███ beats 6/6 | 22% |
| BFCL v3 lab | 63.7% ✎ | ▁▁▁▁▁▃▅█▆▅▄▁ beats 115/170 | 2.7% |
coding▁▁▁▃▆██▇▆▄▃▁ #251 of 458 * · 6.2% of its score · 1 test
Beats Qwen3-VL 32B Instruct, Solar Open 100B (102B-A12B), GLM-4.7-Flash here, though they rank higher overall. Trails QwQ-32B-Preview, InternLM2.5 20B Chat, Mistral Small Instruct 22B (2409) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| BigCodeBench lab | 28.1% ✎ | ▅····▃·█▃▅█▃ beats 3/12 | 6.2% |
math█··········█ #1 of 2 * · 25% of its score · 2 tests
vision & documents▁▁▃▂▁▃▄▂▆▇█▄ #108 of 147 * · 15% of its score · 2 tests
Beats DeepSeek V4 Flash Vision (Exp), GPT-4 Turbo (2024-04-09), Molmo 2 8B here, though they rank higher overall. Trails Gemma 3 27B, Mistral Small 3.1 24B (2503) here, though they rank lower overall.
| test | result | vs the others in its run | counts |
|---|---|---|---|
| CharXiv (reasoning questions) lab | 48.8% ✎ | ▂▅▃▅▅▆▆▅▆█▆▂ beats 27/100 | 8.1% |
| CharXiv descriptive questions lab | 84.6% ✎ | ▂▄▄▅▇▄█▇▇▂▁▂ beats 34/58 | 7.1% |
other tests2 not in a task · 28% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| GPQA Diamond lab | 57.1% ✎ | ▁▁▁▁▂▄▅▅███▄ beats 104/310 | 18% |
| MMLU lab | 87.4% ✎ | ▁▁▁▁▁▂▃▅█▃▆▃ beats 168/183 | 10% |
$ where to run it
Through an API
api · Amazon Bedrock · $5.00/M · $2.5 in / $12.5 out per 1M tokens · 1M context · aws.amazon.com ↗ · price sourceUSD per 1M tokens, Amazon Bedrock Standard tier, geo cross-Region inference and in-Region, Amazon Nova Premier. Page table values are rendered from AWS's own price data file for the page (b0.p.awsstatic.com meteredUnitMaps, bedrock USD, published 2026-09-25T06:55:02Z); US East (N. Virginia) selected. Priority/Flex/Batch tiers, cached tokens and non-text modalities excluded. Currency: USD. AWS official Amazon Nova pricing page, observed 2026-09-25.
$ how the score was worked out · Amazon vendor-reported benchm… 77%, MASSIVE-Agents — Kulkarni et… 22%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 Amazon vendor-reported benchmark scores for its own Nova models (Amazon Science technical reports) · lab-reported | 8 | 77% |
 MASSIVE-Agents — Kulkarni et al. (2025) · lab-reported | 1 | 22% |
| prior · pull toward the middle | 1% | |
| weighted average | = 54.2 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json