whatmodel.sh / Llama-3-8B-ProLong-512k-Instruct

Llama-3-8B-ProLong-512k-Instruct

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Llama-3-8B-ProLong-512k-Instruct is an open-weights language model from Princeton NLP, released August 2024. Its published size is 8.03B parameters. Licence: llama3.

Catalogue identity notes

Maker repo princeton-nlp/Llama-3-8B-ProLong-512k-Instruct @bf92e49: licence tag llama3. 8,030,261,248 parameters from the Hugging Face safetensors metadata (dense, so active = total). Released 2024-08: month the maker's Hugging Face repo was created (createdAt 2024-08-22).

50.2±4.5#338 of 719 LLMsfrom 3 benchmark results across 2 sources

$ how it compares

The LLMs ranked either side of Llama-3-8B-ProLong-512k-Instruct, out of 828 with a score, then more from Princeton NLP.

#modelscorewhereRAMreleased
406Jamba 1.5 Large AI21 Labs50±1local~839GB2024-08
407Gemini 1.5 Flash (001) Google50±1comparison only2024-05
408Llama-3-8B-ProLong-512k-Instruct Princeton NLP50±5local~35GB2024-08
409EXAONE Deep 2.4B LG AI Research50±2local~3.2GB2025-03
410Mellum2 12B-A2.5B Instruct JetBrains50±2local~8.6GB2026-05
more from Princeton NLP
247Gemma 2 9B IT SimPO Princeton NLP59±1local~7.6GB2024-07
248Gemma 2 9B IT DPO (Princeton) Princeton NLP59±1local~7.6GB2024-07
400Llama-3-Instruct-8B-SimPO Princeton NLP51±1local~6.6GB2024-05

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests3 not in a task · 91% of its score
testresultvs the others in its rundifficultycountswhy
Open PT-LLM Leaderboard average (9 Portuguese tasks)68.8% ✎▃▁▁▁▁▁▁▁▂█▅▁ beats 155/276medium · avg 73%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 39%at its level · noisy
RULER 64k83.2% ✎▁▁·▁·▂▃▄█▂·▁ beats 10/22easy · avg 83%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 38%typical
RULER 128k81.6% ✎▄··▂▄▂▆██▄▄▂ beats 18/21easy · avg 68%▇▇▇▇▇▇ 15%at its level · noisy

$ where to run it

Other ways to run it

  1. Run it as published · needs ~32.12 GB · from huggingface.co/princeton-nlp (official)
    uv run --with torch --with 'transformers<5' --with accelerate python -c "from transformers import pipeline; print(pipeline('text-generation', model='princeton-nlp/Llama-3-8B-ProLong-512k-Instruct', torch_dtype='auto', device_map='auto')('Once upon a time', max_new_tokens=40)[0]['generated_text'])"

Official weights and documentation: huggingface.co/princeton-nlp ↗

$ how the score was worked out · NVIDIA RULER repository READM… 52%, Open Portuguese LLM Leaderboa… 39%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
NVIDIA RULER repository README results table2▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 52%Apache-2.0
Open Portuguese LLM Leaderboard (archived)1▇▇▇▇▇▇▇▇▇▇▇▇ 39%Apache-2.0
prior · pull toward the middle▇▇▇ 9%
weighted average= 50.2

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json