whatmodel.sh / Luminous Base (13B)

Luminous Base (13B)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Luminous Base (13B) is a closed-weights language model from Aleph Alpha, released February 2023. Its published size is 13B parameters. Licence: proprietary.

Catalogue identity notes

Aleph Alpha API model luminous-base (13B per Aleph Alpha's 'Luminous Performance Benchmarks' post dated 20/02/2023, which says the models are 'available on the Completion Playground and API client'). 'released' is the date of that first-party post; the Luminous series was sold earlier (2022) but no first-party launch page was fetched. API-only, later deprecated. Comparison-only.

Model information sources

Release date ↗

28.9±12.9comparison only: no current run option verifiedfrom 10 benchmark results across 1 source

Score within the ARC Challenge comparison set only.

$ how it compares

Scored LLMs released closest to Luminous Base (13B), out of 1182 with a score.

#modelscorewhereRAMreleased
931LLaMA 65B Meta40±1comparison only2023-02
986LLaMA 33B Meta37±1comparison only2023-02
1057LLaMA 13B Meta32±1comparison only2023-02
1077LLaMA 7B Meta30±1comparison only2023-02
1118Pythia 12B EleutherAI25±1local~9.5GB2023-02

see all LLMs · find the ones that fit your machine

$ what it's good at

other tests10 not in a task · 87% of its score
testresultvs the others in its rundifficultycountswhy
COPA lab83.0% ✎▁·▁▁▁▅▅█▆▂▁▁ beats 37/67easy · avg 86%▇▇▇▇▇▇▇ 17%sharp · at its level · small run
OpenBookQA lab27.8% ✎▅▁▁▃▂▂▃█▃▁▃▁ beats 22/107very hard · avg 29%▇▇▇▇▇ 12%sharp · too hard for it · small run
TriviaQA lab20.5% ✎▁▁▂▂▄▄▃▆▅█▅▂ beats 23/80hard · avg 28%▇▇▇▇ 10%small run
BoolQ lab68.3% ✎▁····▁▂█▆▅▆▄ beats 47/103medium · avg 70%▇▇▇▇ 10%at its level · small run
ARC Challenge lab37.0% ✎▄▁▁▂▁▄█▇▃▂▃▂ beats 45/140hard · avg 40%▇▇▇ 7.3%small run
WinoGrande lab64.6% ✎▁▁▁▁▂▁▁▅▇█▃▁ beats 56/131hard · avg 67%▇▇▇ 7.2%small run
ARC Easy lab70.3% ✎▁▃▂▄▆█▅▃▁··▁ beats 67/100easy · avg 74%▇▇▇ 6.8%at its level · small run
HellaSwag lab53.3% ✎▁▁▁▂▂▂▄▆█▇▄▂ beats 27/135medium · avg 56%▇▇ 5.8%at its level · small run
2 more tests
testresultvs the others in its rundifficultycountswhy
LAMBADA (OpenAI version), last-word accuracy lab70.2% ✎▁·····▁▁▃▃██ beats 55/86easy · avg 73%▇▇ 5.5%at its level · small run
PIQA lab77.3% ✎▁▁▁▂▂▃▅█▅▃▁▁ beats 65/123medium · avg 79%▇▇ 5.4%at its level · small run

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Aleph Alpha's own Luminous be… 87%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Aleph Alpha's own Luminous benchmark results (Luminous performance benchmarks post) · lab-reported10Factual vendor-reported score (not licensed content)
prior · pull toward the middle
weighted average= 28.9

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json