whatmodel.sh / decision / djev-spark (DecisionBench 1444f3e9)

djev-spark (DecisionBench 1444f3e9)

+ contribute

djev-spark (DecisionBench 1444f3e9) is a source-available decision implementation from mmastrac.

Catalogue identity notes

DecisionBench v1 comparison: mmastrac/djev-spark, revision 1444f3e927f83ba508e5b28a4fd4fdd9ecd0976b; language-model; adapter native-djev-spark-v1-anchored-wide-choice; probability source native-with-anchored-wide-choice-reconstruction. Kept separate from newer or unpinned Decision Index recipes; standard prompt, not compact. No unverified runtime or OpenRouter price imported.

50.7±1.6#197 of 355 decision modelsfrom 27 benchmark results across 1 source

$ how it compares

The decision models ranked either side of djev-spark (DecisionBench 1444f3e9), out of 357 with a score.

#	model	score	where	RAM	released
197	Jev-Omni akhilaaa3	51±1	model files		2026-09-22
198	LiquidAI d1-3B Liquid AI	51±1	model files
199	djev-spark (DecisionBench 1444f3e9) mmastrac	51±2	source code
200	vLLM-SR Decision 2.0 Nox 4B vLLM Semantic Router	51±1	local	~12GB
201	Decider 4B Mapika	51±1	local	~11GB

see all decision models · find the ones that fit your machine

$ what it's good at

workflows▁▁▁▂▁▂▂▆█▄▁▁ #75 of 114 · 99% of its score · 27 tests

Trails Solar Decide Flash, InternLM Intern-Decision Qwen3.5-4B, StartLux-Decision v1.1 0.8B here, though they rank lower overall.

test	result	vs the others in its run	difficulty	counts	why
DecisionBench v1 · action selection (all-request accuracy) submitted	47.3% ✎	▂▁▂▃▄▃▁▃▂█▇▄ beats 25/56	medium · avg 51%	▇▇▇▇▇▇▇▇ 19%	at its level
DecisionBench v1 · routing (all-request accuracy) submitted	66.0% ✎	▁▁▁▁▂▂▁▁▁▁▃█ beats 27/56	medium · avg 64%	▇▇▇▇▇▇ 14%	sharp · at its level
DecisionBench v1 · navigation (all-request accuracy) submitted	59.0% ✎	▁▁·▃▁▁▁▁▁▃▃█ beats 26/56	medium · avg 58%	▇▇▇▇ 11%	sharp · at its level
DecisionBench v1 · record classification (all-request accuracy) submitted	37.0% ✎	▃▂▁▂▂▂▄▁▃▆██ beats 23/56	medium · avg 39%	▇▇▇ 8.4%	at its level
DecisionBench v1 · verification (all-request accuracy) submitted	62.0% ✎	▁▁▅▁▃▂▂▁▁▇▇█ beats 43/56	medium · avg 56%	▇▇ 6.0%	blunt · at its level
DecisionBench v1 · completion assessment (all-request accuracy) submitted	85.0% ✎	▁▁▂▃▄·▂▁▃▆█▂ beats 24/56	very easy · avg 86%	▇▇ 5.0%	sharp
DecisionBench v1 · triage (all-request accuracy) submitted	79.0% ✎	▁·▁·▂▃▄▃▃█▄▂ beats 27/56	easy · avg 78%	▇▇ 4.8%	sharp
DecisionBench v1 · document record classification (all-request accuracy) submitted	42.2% ✎	▂▂·▁▂▄▃▆▄█▁▁ beats 24/56	medium · avg 46%	▇▇ 4.8%	at its level

19 more tests

test	result	vs the others in its run	difficulty	counts	why
DecisionBench v1 · bounded extraction (all-request accuracy) submitted	92.7% ✎	▃▃▂·▁▁▁▁▁▁█▅ beats 28/56	very easy · avg 86%	▇▇ 4.7%	sharp · noisy
DecisionBench v1 · rubric scoring prioritization (all-request accuracy) submitted	48.6% ✎	▁▁▅▅▃▃▅█▃▃·▁ beats 39/56	medium · avg 45%	▇▇ 4.0%	at its level
DecisionBench v1 · content classification (all-request accuracy) submitted	75.0% ✎	▁····▁▁▁▁▁▁█ beats 24/56	easy · avg 73%	▇ 3.5%	at its level
DecisionBench v1 · review scoring (all-request accuracy) submitted	54.0% ✎	▁······▂▃▅▇█ beats 31/56	medium · avg 52%	▇ 2.7%	at its level
DecisionBench v1 · retrieval ranking (all-request accuracy) submitted	58.0% ✎	▁▁·▁▁▂▃▂▃▁▁█ beats 45/56	medium · avg 35%	▇ 2.0%	at its level · noisy
DecisionBench v1 · risk scoring (all-request accuracy) submitted	65.0% ✎	▁▁·▁▂▂▁▃▃█▇▃ beats 36/56	medium · avg 62%	▇ 1.9%	blunt · at its level
DecisionBench v1 · target selection (all-request accuracy) submitted	41.0% ✎	▁▁·▁▁▁▃▁▂▁▁█ beats 36/56	hard · avg 29%	▇ 1.8%	at its level · noisy
DecisionBench v1 · reasoning (all-request accuracy) submitted	46.1% ✎	▂·▁▁▆█▅▃▃▂▁▁ beats 31/56	medium · avg 52%	▇ 1.2%	at its level
DecisionBench v1 · retrieval verification (all-request accuracy) submitted	61.1% ✎	█▁▃▁▁▁▁▁▂▅▁▁ beats 26/56	easy · avg 74%	▇ 0.8%	at its level · noisy
DecisionBench v1 · document workflows (all-request accuracy) submitted	51.1% ✎	█▄▃▁▁▃▂▂▁·▁▁ beats 21/56	easy · avg 66%	▇ 0.7%	blunt · at its level
DecisionBench v1 · guardrails moderation (all-request accuracy) submitted	38.3% ✎	▁▁▁▂▆▃▃█···▁ beats 12/56	medium · avg 52%	▇ 0.5%	blunt · at its level · noisy
DecisionBench v1 · safety gating (all-request accuracy) submitted	65.0% ✎	▂·····▆█▂▂▄▇ beats 26/56	easy · avg 77%	▇ 0.5%	noisy
DecisionBench v1 · document classification (all-request accuracy) submitted	34.0% ✎	▂▁·▂▁▁▁▄▁▂▂█ beats 43/56	hard · avg 21%	▇ 0.3%	blunt · noisy
DecisionBench v1 · semantic filtering (all-request accuracy) submitted	76.0% ✎	▁·▁▁·▁▁█▂▃█▂ beats 28/56	easy · avg 71%	▇ 0.3%	blunt · at its level · noisy
DecisionBench v1 · content moderation (all-request accuracy) submitted	90.0% ✎	▁····▃▅▃▃▂·█ beats 31/56	easy · avg 82%	▇ 0.2%	noisy
DecisionBench v1 · context management (all-request accuracy) submitted	67.0% ✎	▁·····▁▇▆▁█▃ beats 25/56	easy · avg 73%	▇ 0.2%	blunt · at its level · noisy
DecisionBench v1 · entity alignment (all-request accuracy) submitted	97.7% ✎	▁·▁▄▇▄▅▄▂▁▄█ beats 41/56	easy · avg 66%	▇ 0.2%	at its level · noisy
DecisionBench v1 · function agent skill routing (all-request accuracy) submitted	93.2% ✎	▃▃▄▅▆▂▅▄▃▄█▃ beats 37/56	easy · avg 68%	▇ 0.1%	blunt · at its level · noisy
DecisionBench v1 · routing triage (all-request accuracy) submitted	57.4% ✎	█▁···▁·▂▆▅▂▂ beats 34/56	medium · avg 42%	▇ 0.0%	blunt · at its level · noisy

$ where to run it

Implementation and setup instructions ↗
Published implementation and setup documentation verified. This entry describes a decision readout or evaluated configuration, not necessarily separate trained weights. Follow the referenced version and model settings; hardware compatibility has not been verified. Source · checked 2026-10-31.

Model repository and documentation: github.com/mmastrac ↗

$ how the score was worked out · Hanno Labs DecisionBench v1 r… 99%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

source	results	share of the score	licence
Hanno Labs DecisionBench v1 reviewed results · submitted; independent execution unverified	27	▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 99%	CC0 1.0
prior · pull toward the middle		▇ 1%
weighted average		= 50.7

$ improve this page

Update model information Add a place to run it Add a test result Propose a benchmark Correct a benchmark score Edit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated 2026-11-03 02:37 UTC · sources and licences · json