whatmodel.sh / Luminous Extended (30B)

Luminous Extended (30B)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Luminous Extended (30B) is a closed-weights language model from Aleph Alpha, released February 2023. Its published size is 30B parameters. Licence: proprietary.

Catalogue identity notes

Aleph Alpha API model luminous-extended (30B per Aleph Alpha's 'Luminous Performance Benchmarks' post dated 20/02/2023, which says the models are 'available on the Completion Playground and API client'). 'released' is the date of that first-party post; the Luminous series was sold earlier (2022) but no first-party launch page was fetched. API-only, later deprecated. Comparison-only.

Model information sources

Release date ↗

58.9±13.0comparison only: no current run option verifiedfrom 10 benchmark results across 1 source

Score within the ARC Challenge comparison set only.

$ how it compares

Scored LLMs released closest to Luminous Extended (30B), out of 1182 with a score.

#modelscorewhereRAMreleased
931LLaMA 65B Meta40±1comparison only2023-02
986LLaMA 33B Meta37±1comparison only2023-02
1057LLaMA 13B Meta32±1comparison only2023-02
1077LLaMA 7B Meta30±1comparison only2023-02
1118Pythia 12B EleutherAI25±1local~9.5GB2023-02

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests10 not in a task · 86% of its score
testresultvs the others in its rundifficultycountswhy
COPA lab84.0% ✎▁·▁▁▁▅▅█▆▂▁▁ beats 40/67easy · avg 86%▇▇▇▇▇▇▇ 17%sharp · small run
OpenBookQA lab29.4% ✎▅▁▁▁▃▂▃█▃▁▃▁ beats 25/107very hard · avg 29%▇▇▇▇▇ 11%sharp · small run
BoolQ lab69.1% ✎▁····▁▂█▆▅▆▄ beats 49/103medium · avg 70%▇▇▇▇ 10%at its level · small run
TriviaQA lab28.7% ✎▁▁▂▂▄▄▃▆▅█▅▂ beats 28/80hard · avg 28%▇▇▇▇ 10%at its level · small run
WinoGrande lab67.3% ✎▁▁▁▁▂▁▁▅▇█▃▁ beats 74/131hard · avg 67%▇▇▇ 7.2%at its level · small run
ARC Challenge lab40.7% ✎▄▁▁▂▁▄█▇▃▂▃▂ beats 53/140hard · avg 40%▇▇▇ 7.1%small run
ARC Easy lab75.0% ✎▁▃▂▄▆█▅▃▁··▁ beats 77/100easy · avg 74%▇▇▇ 6.9%at its level · small run
HellaSwag lab56.9% ✎▁▁▁▂▂▂▄▆█▇▄▂ beats 34/135medium · avg 56%▇▇ 5.8%at its level · small run
2 more tests
testresultvs the others in its rundifficultycountswhy
LAMBADA (OpenAI version), last-word accuracy lab72.5% ✎▁·····▁▁▃▃██ beats 67/86easy · avg 73%▇▇ 5.6%small run
PIQA lab78.8% ✎▁▁▁▂▂▃▅█▅▃▁▁ beats 82/123medium · avg 79%▇▇ 5.5%at its level · small run

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Aleph Alpha's own Luminous be… 86%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Aleph Alpha's own Luminous benchmark results (Luminous performance benchmarks post) · lab-reported10Factual vendor-reported score (not licensed content)
prior · pull toward the middle
weighted average= 58.9

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json