whatmodel.sh / Jamba 1.5 Large

Jamba 1.5 Large

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Jamba 1.5 Large is an open-weights language model from AI21 Labs, released August 2024. Its published size is 398B parameters (94B active per token). Licence: jamba-open-model-license.

Catalogue identity notes

Comparison-only. Arena entry 'jamba-1.5-large' (text). LMArena's leaderboard table (arena-leaderboard space @6f2eeeb) labels 'jamba-1.5-large' as 'Jamba-1.5-Large' (AI21 Labs, Jamba Open), linking https://www.ai21.com/jamba. Maker repo ai21labs/AI21-Jamba-Large-1.5 @57d0bbc: licence tag other (jamba-open-model-license). Parameters: AI21 docs: Jamba Large '398B parameters (94B active)' (HF safetensors metadata 398.6B). Released 2024-08: month the maker's Hugging Face repo was created (createdAt 2024-08-19). AI21 API snapshot jamba-large-1.5-2024-08, deprecated 2025-05-06 (AI21 model deprecation table).

50.3±1.2#337 of 719 LLMsfrom 12 benchmark results across 3 sources

$ how it compares

The LLMs ranked either side of Jamba 1.5 Large, out of 828 with a score, then more from AI21 Labs.

#modelscorewhereRAMreleased
404OpenThinker3 7B Open Thoughts50±1local~6GB2025-05
405CodeQwen1.5 7B Chat Alibaba50±1local~6.4GB2024-04
406Jamba 1.5 Large AI21 Labs50±1local~839GB2024-08
407Gemini 1.5 Flash (001) Google50±1comparison only2024-05
408Llama-3-8B-ProLong-512k-Instruct Princeton NLP50±5local~35GB2024-08
more from AI21 Labs
491Jamba 1.5 Mini AI21 Labs46±1local~34GB2024-08

see all LLMs · find the ones that fit your machine

$ what it's good at

agent tasks▁▁▃▆█▆▅▅▅▄▃▁ #111 of 300 * · 2.3% of its score · 1 test

Beats o3-pro, Qwen3.6 Plus, o3 here, though they rank higher overall. Trails SmolLM3 3B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
BFCL (version not stated) lab85.5% ✎▄▆▄▂·█▆·▄··▂ beats 14/17easy · avg 81%▇ 2.3%blunt · small run
coding█··········█ #1 of 2 * · 8.4% of its score · 1 test
testresultvs the others in its rundifficultycountswhy
HumanEval lab71.3% ✎▁▁▁▁▁▃▅▅█▃▂▁ beats 123/190easy · avg 74%▇▇▇ 8.4%sharp · at its level · small run
chat & writing▁▁▁▃▅▄▅█▆▆▇▃ #233 of 393 · 19% of its score · 3 tests

Beats Mistral Small 4 (119B-A6.5B), Ministral 3 14B Reasoning 2512, Qwen2.5 Coder 32B Instruct here, though they rank higher overall. Trails Flan-T5 XXL, LFM2.5 1.2B Instruct, Gemma 3 4B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
LMArena (text)1289.4 ✎▁▁▁▂▃▃▃▆▆▇█▃ beats 121/379avg 1268▇▇▇▇▇ 12%blunt
Arena-Hard (version/judge unspecified) lab65.4% ✎▃▃▂▅▃▆▅█▅▅▂▃ beats 15/30medium · avg 58%▇▇ 4.3%at its level · small run
IFEval (aggregation unspecified) lab81.5% ✎▁▁▃▁▂▄▄▆█▃▂▁ beats 56/141easy · avg 80%▇ 2.9%small run
math▁▂▃▅▆▅▄█▄▁▁▁ #249 of 398 * · 9.5% of its score · 1 test

Beats GPT-4 Turbo (2024-04-09), Molmo 2 8B, Llama 3.3 70B Instruct here, though they rank higher overall. Trails DeepSeek R1 Distill Qwen 1.5B, Tulu 2 DPO 70B, OpenThinker 7B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
GSM8K lab87.0% ✎▂▂▂▅▅▆▅▇█▇▄▃ beats 111/146easy · avg 85%▇▇▇▇ 9.5%sharp · small run
other tests5 not in a task · 60% of its score
testresultvs the others in its rundifficultycountswhy
MMLU-Pro +149.5% ✎▃▂▅█▃·█▃▅··▂ beats 12/23medium · avg 57%▇▇▇▇▇▇▇▇▇▇▇ 28%at its level · several runs
BIG-Bench Hard lab65.5% ✎▁▁▁▁▂▄▅▅█▃▄▁ beats 76/129medium · avg 65%▇▇▇▇▇ 11%sharp · at its level · small run
GPQA (subset unspecified) lab36.9% ✎▂▁▂▂▄▃▆▃▅█▄▃ beats 32/104hard · avg 37%▇▇▇▇ 11%sharp · small run
MMLU lab80.0% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 135/183easy · avg 77%▇▇▇ 7.6%at its level · small run
ARC Challenge lab93.0% ✎▆▁▁▂▂▅▇█▄▃▃▃ beats 96/97easy · avg 81%▇ 2.0%sharp · noisy · small run

$ where to run it

Other ways to run it

  1. Run it as published · needs ~797.11 GB, NVIDIA only · from huggingface.co/ai21labs (official)
    pip install 'vllm>=0.5.5' && vllm serve ai21labs/AI21-Jamba-Large-1.5 --tensor-parallel-size 8 --max-model-len 225280 --quantization experts_int8

Official weights and documentation: huggingface.co/ai21labs ↗

$ how the score was worked out · Jamba-1.5: Hybrid Transformer… 66%, MMLU-Pro author-released eval… 21%, LMArena leaderboard dataset 12%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Jamba-1.5: Hybrid Transformer-Mamba Models at Scale (AI21, arXiv 2408.12570v1) · lab-reported10▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 66%CC BY-SA 4.0
MMLU-Pro author-released evaluation runs1▇▇▇▇▇▇ 21%Apache-2.0
LMArena leaderboard dataset1▇▇▇▇ 12%CC BY 4.0
prior · pull toward the middle▇ 1%
weighted average= 50.3

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json