whatmodel.sh / Llama-3-8B-Instruct-SkillMix

Llama-3-8B-Instruct-SkillMix

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Llama-3-8B-Instruct-SkillMix is an open-weights language model from Princeton PLI, released September 2024. Licence: llama3.

Catalogue identity notes

Maker repo PrincetonPLI/Llama-3-8B-Instruct-SkillMix @b809eb5: licence tag llama3. Released 2024-09: month the maker's Hugging Face repo was created (createdAt 2024-09-08). Comparison-only here: no run command was verified for this checkpoint (no weight files in the repo listing).

51.6±1.6comparison only: no current run option verifiedfrom 1 benchmark result across 1 source* thinly tested, treat the rank as provisional

$ how it compares

The LLMs ranked either side of Llama-3-8B-Instruct-SkillMix, out of 1132 with a score.

#modelscorewhereRAMreleased
525Qwen2.5 Coder 7B Instruct Alibaba52±1local~6GB2024-09
526Jamba2 Mini AI21 Labs52±15api price unknown +local~32GB2026-01
527Llama-3-8B-Instruct-SkillMix Princeton PLI52±2comparison only2024-09
528Qwen2.5 14B Instruct Alibaba52±1local~10GB2024-09
529Gemini Experimental 1121 Google52±2comparison only2024-11

see all LLMs · find the ones that fit your machine

$ what it's good at

chat & writing▁▁▁▃▅▅▆█▆▆▆▂ #340 of 579 * · 99% of its score · 1 test

Beats Mistral Small 4 (119B-A6.5B), Ministral 3 14B Reasoning 2512, Qwen2.5 Coder 32B Instruct here, though they rank higher overall. Trails Flan-T5 XXL, Granite 4.0 H 1B, LFM2.5 1.2B Instruct here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
AlpacaEval 2 length-controlled42.8% ✎▁▂▂▃▆▆█▆▄▄▁▁ beats 99/122medium · avg 39%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 99%at its level

$ where to run it

No verified API or download recorded yet · add one

Official weights and documentation: huggingface.co/PrincetonPLI ↗

$ how the score was worked out · AlpacaEval 1.0 and 2.0 leader… 99%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
AlpacaEval 1.0 and 2.0 leaderboards (tatsu-lab/alpaca_eval repository)1▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 99%Apache-2.0
prior · pull toward the middle▇ 1%
weighted average= 51.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json