whatmodel.sh / decision / Decision-1.0-Eos-0.8B (DecisionBench 3c2d6326)

Decision-1.0-Eos-0.8B (DecisionBench 3c2d6326)

+ contribute

Decision-1.0-Eos-0.8B (DecisionBench 3c2d6326) is an open-weights decision model from llm-semantic-router. Licence: apache-2.0.

Catalogue identity notes

DecisionBench v1 comparison: llm-semantic-router/Decision-1.0-Eos-0.8B, revision 3c2d632609ceb66f3a13bbc5f77f3ab8cdeebcdd; decision-model; adapter jev-compatible-server-v1; probability source native_probability_distribution. Kept separate from newer or unpinned Decision Index recipes; standard prompt, not compact. No unverified OpenRouter price imported. Current repository metadata checked separately; an unpinned alias is not evidence that today’s weights reproduce this historical run. Reviewed Ollaya local route added 2026-10-31: decision:eos, manifest fc119d7067ecd029a5316442cf42a394236fff8758ab296f3d25a1285a34e047, matching the independent accuracy-run manifest. Native family implementation; published runtime precision label F32. Memory is measured peak GPU increase (5910 MiB, rounded upward to 5.8 GiB) on RTX 5090 during the short five-question triage workload, not download size or a maximum-context guarantee. Runtime installation is required; no fresh execution performed here. Historical benchmark adapter and input protocol still vary by result.

Model information sources

· Weights licence ↗

40.9±1.3#261 of 355 decision modelsfrom 41 benchmark results across 2 sources

$ how it compares

The decision models ranked either side of Decision-1.0-Eos-0.8B (DecisionBench 3c2d6326), out of 357 with a score.

#	model	score	where	RAM
261	mini-jev Qwen3-4B (decision readout) r-ms	41±1	source code
262	open-jev (pngwn) pngwn	41±1	model files
263	Decision-1.0-Eos-0.8B (DecisionBench 3c2d6326) llm-semantic-router	41±1	local	~5.8GB
264	Decider 2B Mapika	41±1	model files
265	FLock this-that 1.2 FLock.io	40±1	model files

see all decision models · find the ones that fit your machine

$ what it's good at

knowledge & reasoning▁▂▃▂▂▃▃▅▅▆█▂ #237 of 263 · 8.7% of its score · 5 tests

Trails laya (DecisionBench 1c5edc17), JevK5 Lite, LFM2.5-2.6B-RLCD here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
Ollaya · PubMedQA	37.2% ✎	▃▂▃▃·▂▃▃▆█▆▂ beats 0/25	easy · avg 69%	▇ 3.0%	at its level
Ollaya · BoolQ	68.3% ✎	▁··▃▃▁▄▁▃█▇▄ beats 1/25	easy · avg 84%	▇ 2.6%	typical
Ollaya · SQuAD 2 answerability	87.3% ✎	▁▁▃▁▇█▃▃▄▃·▁ beats 23/25	easy · avg 80%	▇ 1.2%	blunt · at its level
Ollaya · VitaminC verification	62.4% ✎	▁···▃·▄█▃··▁ beats 1/25	easy · avg 78%	▇ 1.2%	blunt · at its level
Ollaya · MultiNLI	76.9% ✎	▂··▄██▂▆▆█▆▂ beats 1/25	very easy · avg 87%	▇ 0.8%	blunt

workflows▁▁▁▂▁▂▃▆█▄▁▁ #84 of 114 · 91% of its score · 36 tests

Beats Qwen3-Reranker-4B (DecisionBench 22e68366) here, though they rank higher overall. Trails LiquidAI d1-omni-600M, InternLM Intern-Decision Qwen3.5-0.8B, InternLM Intern-Decision Qwen3.5-2B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
DecisionBench v1 · action selection (all-request accuracy) submitted	37.7% ✎	▂▁▂▃▄▃▃▁▂█▇▄ beats 22/56	medium · avg 51%	▇▇▇▇▇ 12%	at its level
DecisionBench v1 · routing (all-request accuracy) submitted	59.3% ✎	▁▁▁▁▂▂▁▁▁▃▃█ beats 24/56	medium · avg 64%	▇▇▇▇ 10%	sharp · at its level
DecisionBench v1 · navigation (all-request accuracy) submitted	58.0% ✎	▁▁·▃▁▁▁▁▁▃▃█ beats 25/56	medium · avg 58%	▇▇▇ 7.5%	sharp · at its level
DecisionBench v1 · bounded extraction (all-request accuracy) submitted	89.9% ✎	▃▃▂·▁▁▁▁▁▃█▅ beats 25/56	very easy · avg 86%	▇▇ 5.4%	sharp · at its level · noisy
DecisionBench v1 · completion assessment (all-request accuracy) submitted	75.0% ✎	▁▁▂▃▄·▃▁▁▆█▂ beats 20/56	very easy · avg 86%	▇▇ 5.3%	sharp
DecisionBench v1 · record classification (all-request accuracy) submitted	27.0% ✎	▃▂▁▂▂▂▄▁▃▆██ beats 17/56	medium · avg 39%	▇▇ 5.0%	at its level
Ollaya · Typed decisions test	51.2% ✎	▄·▂▆▄▄▂·▄█▆▆ beats 7/22	medium · avg 64%	▇▇ 4.5%	at its level
DecisionBench v1 · triage (all-request accuracy) submitted	66.0% ✎	▁·▁·▂▃▄▃▂█▄▂ beats 23/56	easy · avg 78%	▇▇ 4.2%	sharp · at its level

28 more tests

test	result	vs the others in its run	difficulty	counts	why
DecisionBench v1 · verification (all-request accuracy) submitted	56.5% ✎	▁▁▅▁▃▂▂▁▁▇▇█ beats 25/56	medium · avg 56%	▇▇ 4.1%	blunt · at its level
Ollaya · PAWS paraphrase	67.2% ✎	▂▂▂▃·▃█▆·▂·▁ beats 6/25	easy · avg 81%	▇ 3.2%	at its level
Ollaya · MASSIVE German intent	75.4% ✎	▁····▁·▂▃▆█▃ beats 8/23	easy · avg 84%	▇ 3.2%	sharp · at its level
DecisionBench v1 · document record classification (all-request accuracy) submitted	40.4% ✎	▂▂·▁▂▄▃▆▄█▁▁ beats 22/56	medium · avg 46%	▇ 3.0%	at its level
DecisionBench v1 · content classification (all-request accuracy) submitted	73.0% ✎	▁····▁▁▁▁▁▁█ beats 23/56	easy · avg 73%	▇ 2.8%	at its level
Ollaya · Aegis 2 safety	68.8% ✎	▂·▁▁▂·▃▃▂█▃▁ beats 6/25	easy · avg 77%	▇ 2.6%	at its level
DecisionBench v1 · rubric scoring prioritization (all-request accuracy) submitted	35.6% ✎	▁▁▅▅▃▃▅█▃▃·▁ beats 21/56	medium · avg 45%	▇ 2.5%	at its level
Ollaya · SummEval relevance	22.1% ✎	▁··▁▂▁▅··▃██ beats 5/25	medium · avg 41%	▇ 2.0%	at its level
DecisionBench v1 · review scoring (all-request accuracy) submitted	57.0% ✎	▁······▂▃▅▇█ beats 36/56	medium · avg 52%	▇ 1.8%	at its level
Ollaya · SummEval consistency	29.2% ✎	▁·▁▁··▃▁▁▁▄█ beats 3/25	easy · avg 75%	▇ 1.6%	sharp · at its level · noisy
Ollaya · MASSIVE English intent	81.7% ✎	▁·······▂▃█▃ beats 7/23	very easy · avg 86%	▇ 1.6%	noisy
DecisionBench v1 · risk scoring (all-request accuracy) submitted	61.0% ✎	▁▁·▁▂▂▁▃▃█▇▃ beats 25/56	medium · avg 62%	▇ 1.3%	blunt · at its level
DecisionBench v1 · retrieval ranking (all-request accuracy) submitted	46.0% ✎	▁▁·▁▁▂▃▂▃▁▃█ beats 34/56	medium · avg 35%	▇ 1.1%	noisy
DecisionBench v1 · target selection (all-request accuracy) submitted	40.0% ✎	▁▁·▁▁▁▃▁▂▁▁█ beats 35/56	hard · avg 29%	▇ 1.0%	noisy
Ollaya · HelpSteer 2 scoring	25.7% ✎	▁▁··▃·▁▃▇█▅▄ beats 2/23	medium · avg 38%	▇ 0.8%	blunt · at its level
DecisionBench v1 · reasoning (all-request accuracy) submitted	37.1% ✎	▂·▁▁▆█▅▃▃▂▁▁ beats 16/56	medium · avg 52%	▇ 0.8%	at its level
DecisionBench v1 · retrieval verification (all-request accuracy) submitted	63.1% ✎	█▁▃▁▁▁▁▁▂▅▁▁ beats 28/56	easy · avg 74%	▇ 0.6%	at its level · noisy
DecisionBench v1 · document workflows (all-request accuracy) submitted	59.0% ✎	█▄▃▁▁▃▂▂▁·▁▁ beats 32/56	easy · avg 66%	▇ 0.5%	blunt · at its level
DecisionBench v1 · safety gating (all-request accuracy) submitted	73.0% ✎	▂·····▆█▃▂▄▇ beats 31/56	easy · avg 77%	▇ 0.4%	at its level · noisy
DecisionBench v1 · guardrails moderation (all-request accuracy) submitted	46.1% ✎	▁▁▁▂▆▃▃█···▁ beats 27/56	medium · avg 52%	▇ 0.3%	blunt · at its level · noisy
DecisionBench v1 · semantic filtering (all-request accuracy) submitted	52.0% ✎	▁·▁▁·▁▁█▂▃█▂ beats 13/56	easy · avg 71%	▇ 0.2%	blunt · at its level · noisy
DecisionBench v1 · document classification (all-request accuracy) submitted	26.0% ✎	▂▁·▂▁▁▁▄▁▂▃█ beats 34/56	hard · avg 21%	▇ 0.2%	blunt · noisy
DecisionBench v1 · content moderation (all-request accuracy) submitted	81.0% ✎	▁····▃▅▃▂▂·█ beats 24/56	easy · avg 82%	▇ 0.2%	noisy
Ollaya · Civil Comments	65.3% ✎	▂▃▂·▂▅▅▆▃▅·█ beats 1/23	very easy · avg 86%	▇ 0.1%	blunt · noisy
DecisionBench v1 · context management (all-request accuracy) submitted	59.0% ✎	▁·····▁▇▆▁█▃ beats 19/56	easy · avg 73%	▇ 0.1%	blunt · at its level · noisy
DecisionBench v1 · entity alignment (all-request accuracy) submitted	76.5% ✎	▁·▁▄▇▄▅▄▂▁▄█ beats 30/56	easy · avg 66%	▇ 0.1%	at its level · noisy
DecisionBench v1 · function agent skill routing (all-request accuracy) submitted	97.4% ✎	▃▃▄▅▆▂▅▄▃▄█▃ beats 50/56	easy · avg 68%	▇ 0.1%	blunt · at its level · noisy
DecisionBench v1 · routing triage (all-request accuracy) submitted	73.9% ✎	█▁···▁·▂▆▅▂▂ beats 47/56	medium · avg 42%	▇ 0.0%	blunt · at its level · noisy

$ where to run it

Other ways to run it

Run it as published · needs 5.8 GB, NVIDIA only · from github.com/ollaya-dev
ollaya run decision:eos --preset triage "The customer requests a refund."

Download model files ↗
Published model files verified. This is the revision recorded by the benchmark. A matching Ollaya runtime manifest and published short-workload GPU memory measurement were reviewed separately; see local variants and identity_note. The command was not executed in this review. Source · checked 2026-10-31.

Model repository and documentation: huggingface.co/llm-semantic-router ↗

$ how the score was worked out · Hanno Labs DecisionBench v1 r… 71%, Ollaya · original runs 28%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Hanno Labs DecisionBench v1 reviewed results · submitted; independent execution unverified	27	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 71%	CC0 1.0
Ollaya · original runs	14	▇▇▇▇▇▇▇▇▇ 28%	Apache-2.0
prior · pull toward the middle		▇ 1%
weighted average		= 40.9

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-11-03 02:37 UTC · sources and licences · json