whatmodel.sh / decision / RSI-Jev v3.0 2B (JevBench mini)

RSI-Jev v3.0 2B (JevBench mini)

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

RSI-Jev v3.0 2B (JevBench mini) is a source-available decision implementation from Shanghua Gao.

Catalogue identity notes

JevBench 0.1.0 mini tested configuration rsi-jev-2b: Decoder, trained scorer head; base Qwen3.5-2B-Base; readout trained readout/scorer head; SFT then RL, fitted calibration. Separate identity because the report does not establish equivalence to the newer Decision Index checkpoint or serving recipe. No latency imported.

45.2±2.3#241 of 355 decision modelsfrom 6 benchmark results across 1 source

$ how it compares

The decision models ranked either side of RSI-Jev v3.0 2B (JevBench mini), out of 357 with a score.

#modelscorewhereRAMreleased
241TokenRhythm NeoHorse-Jev-4B TokenRhythm46±1model files2026-09-25
242SemIf Qwen3.5-4B Base (decision readout) TheoLeeCJ46±1source code
243RSI-Jev v3.0 2B (JevBench mini) Shanghua Gao45±2source code
244Metask-Jev-4B Metask-AI-Lab45±1model files
245Jobe Qwen3.5-4B (decision readout) MantisShrimpdev45±1source code

see all decision models · find the ones that fit your machine

$ what it's good at

consistency▃·▂▂·▅▃·▃▅▃█ #13 of 21 · 60% of its score · 5 tests

Beats Kev-4B (JevBench mini) here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
JevBench mini 0.1.0 · Probability measure coherence (pass rate)32.1% ✎▄·▁·▃▃▃·▃▄▁█ beats 8/20medium · avg 43%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 42%sharp · at its level
JevBench mini 0.1.0 · Representation consistency (pass rate)73.1% ✎▃·▂·▂▂▂▃▅▅▃█ beats 6/20easy · avg 79%▇▇▇▇▇▇ 14%at its level
JevBench mini 0.1.0 · Logical coherence (pass rate)73.8% ✎▃·▃·█▅▅██▃█▅ beats 8/20easy · avg 78%▇ 2.3%blunt
JevBench mini 0.1.0 · Choice-set coherence (pass rate)81.9% ✎▁▃▄█▄▄·····▄ beats 13/20easy · avg 79%▇ 1.2%noisy
JevBench mini 0.1.0 · Batch independence (pass rate)100.0% ✎▁··········█ beats 10/20very easy · avg 97%▇ 0.0%blunt · maxed out · noisy
other tests1 not in a task · 38% of its score
testresultvs the others in its rundifficultycountswhy
JevBench mini 0.1.0 · unchanged-answer accuracy85.2% ✎▁·▂·▂···▂▃▁█ beats 9/20very easy · avg 87%▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 38%sharp

$ where to run it

Implementation and setup instructions ↗
Published implementation and setup documentation verified. This entry describes a decision readout or evaluated configuration, not necessarily separate trained weights. Follow the referenced version and model settings; hardware compatibility has not been verified. Source · checked 2026-10-31.

Model repository and documentation: github.com/Shanghua-Gao ↗

$ how the score was worked out · JevBench 0.1.0 mini evaluatio… 98%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
JevBench 0.1.0 mini evaluation reports6Apache-2.0
prior · pull toward the middle
weighted average= 45.2

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json