whatmodel.sh / LLaMA 65B

LLaMA 65B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

LLaMA 65B is an open-weights language model from Meta, released February 2023. Its published size is 65.2B parameters. Licence: llama-1-noncommercial-research.

Catalogue identity notes

Original LLaMA (v1) base model (Touvron et al. 2023), released by Meta to approved researchers under a non-commercial licence; no official public download exists, so no run command is listed (community re-uploads such as huggyllama/* are unofficial).

38.0±1.1comparison only: no current run option verifiedfrom 15 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of LLaMA 65B, out of 828 with a score, then more from Meta.

#modelscorewhereRAMreleased
654Phi-3-mini-4k-instruct (Apr 2024) Microsoft38±2local~4GB2024-04
655Apertus 8B Instruct (2509) Swiss AI38±4local~6.3GB2025-08
656LLaMA 65B Meta38±1comparison only2023-02
657Granite 3B Code Instruct (2K) IBM38±2local~3.7GB2024-04
658DeepSeek-V2 Lite Chat (16B-A2.4B) DeepSeek38±1local~11GB2024-05
more from Meta
24Muse Spark 1.3 Meta77±1api $2.00/M2026-09
40Muse Spark 1.2 Meta75±1api $2.00/M2026-08
43Muse Spark 1.1 Meta75±1api $2.00/M2026-07

see all LLMs · find the ones that fit your machine

$ what it's good at

coding▁▁▁▃▅▇█▇▅▄▃▁ #482 of 524 * · 20% of its score · 2 tests

Beats DeepSeek R1 Distill Qwen 7B, Qwen3.5 2B, DeepSeek R1 Distill Llama 8B here, though they rank higher overall. Trails Llama 3.2 1B Instruct, SmolLM2 1.7B Instruct, Mistral 7B Instruct v0.1 here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
HumanEval lab23.7% ✎▁▁▁▁▁▃▅▅█▃▂▁ beats 24/190easy · avg 66%▇▇▇▇▇ 13%sharp · at its level
MBPP lab37.7% ✎▁▁▁▄▅█▄▅▃▁▂▁ beats 12/81medium · avg 63%▇▇▇ 6.7%at its level
math▁▂▃▅▇▅▅█▄▁▁▁ #386 of 462 * · 19% of its score · 2 tests

Beats GPT-3.5 Turbo (0125), Qwen2-VL 7B Instruct, DBRX Instruct here, though they rank higher overall. Trails Qwen3 0.6B, Gemma 7B (base), DeepSeek R1 Distill Qwen 1.5B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
MATH lab10.6% ✎▃▅▂▆▅▃▅▃█▃▃▂ beats 8/29hard · avg 27%▇▇▇▇ 10%typical
GSM8K lab50.9% ✎▂▂▂▅▅▆▅▇█▇▄▃ beats 48/146easy · avg 76%▇▇▇▇ 9.1%sharp · at its level
other tests11 not in a task · 61% of its score
testresultvs the others in its rundifficultycountswhy
Factual knowledge recall, Premium2k exact match (zero-shot)35.3% ✎▅····▃▅·██·█ beats 12/13medium · avg 59%▇▇▇▇ 10%at its level
ARC Challenge lab56.0% ✎▆▁▁▂▂▅▇█▄▃▃▂ beats 65/97easy · avg 86%▇▇▇ 8.3%sharp · at its level
Factual knowledge recall, Premium2k exact match (10-shot)52.4% ✎▃▅▃··▅▃·█▃▃▅ beats 13/13easy · avg 67%▇▇▇ 6.3%at its level
WinoGrande lab77.0% ✎▁···▂▁▁▅█▇▃▁ beats 78/87easy · avg 90%▇▇▇ 6.3%at its level
MMLU lab63.4% ✎▁▁▁▁▁▂▃▅█▃▆▃ beats 79/183easy · avg 79%▇▇ 6.0%at its level
BoolQ lab85.3% ✎▁▄▅▆▂▆▄▄█▃▄▄ beats 52/58very easy · avg 93%▇▇ 6.0%at its level
OpenBookQA lab60.2% ✎█▁▁▁▄▃▃▇▃▁▄▂ beats 54/65easy · avg 80%▇▇ 5.6%at its level
HellaSwag lab84.2% ✎▁▁▁▁▂▂▃▇█▆▅▃ beats 74/85very easy · avg 92%▇▇ 4.4%typical
3 more tests
testresultvs the others in its rundifficultycountswhy
ARC Easy lab78.9% ✎▁▂▃▂▃▅▇█▇▆▄▆ beats 58/65very easy · avg 90%▇ 3.2%at its level
Social IQA lab52.3% ✎▂·▂██▅▅▁▂▃▆▁ beats 35/51medium · avg 65%▇ 2.9%blunt · at its level
PIQA lab82.8% ✎▁▁▁▃▃▄▇█▄▃▁▁ beats 75/83easy · avg 88%▇ 2.0%blunt · at its level

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · LLaMA: Open and Efficient Fou… 83%, Towards a Holistic Evaluation… 16%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
LLaMA: Open and Efficient Foundation Language Models — authors' own runs · lab-reported13▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 83%CC BY 4.0
Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall (arXiv:2404.16164v1)2▇▇▇▇▇ 16%CC-BY-4.0
prior · pull toward the middle▇ 1%
weighted average= 38.0

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json