whatmodel.sh / decision / Clef

Clef

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Clef is an open-weights decision model from Cloudflare, released October 2026. Its published size is 27.357B parameters. Licence: apache-2.0.

Catalogue identity notes

Exact Decision Index 0.4.1 text-board engine clef. Produces probabilities over supplied choices, not conversational text. Only public per-benchmark results are used; no Full index, private scores or latency is treated as an ability measurement. Model and runtime documentation reviewed 2026-10-31; no fresh hardware execution.

Model information sources

Release date ↗ · Weights licence ↗ · Parameter count ↗

HF safetensors metadata for full weights. Memory estimate: BF16 weights plus 20% runtime allowance and 2 GB; not a measured requirement.

63.2±0.6#74 of 355 decision modelsfrom 149 benchmark results across 10 sources

$ how it compares

The decision models ranked either side of Clef, out of 357 with a score, then more from Cloudflare.

#modelscorewhereRAMreleased
72Xor 26B-A4B v1.2 Juspay Technologies63±1model files2026-10-16
73Senova-4B v2 WangSenQWQ63±1model files
74Clef Cloudflare63±1api $0.24/M input tokens +local~68GB2026-10
75linne-fern 1.0 (27B) MISA-AICORE63±1model files
76Microsoft-Decision-1 Microsoft63±1api price unknown2026-10-09
more from Cloudflare
121Clef-omni Cloudflare59±1api $0.15/M input tokens +local~87GB2026-10
134Clef-flash Cloudflare58±1api $0.038/M input tokens +local~25GB2026-10

see all decision models · find the ones that fit your machine

$ what it's good at

tools & automation▁▃▁▁▁▂▃▄▆██▆ #3 of 250 · 2.1% of its score · 8 tests

Beats Jeffan-31b-1710, baselabs mammoth-1 (27B), Sieve 31B v3 here, though they rank higher overall. Trails Clef-flash, Clef-omni here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Decision Index 0.4.1: API-Bank (accuracy) submitted91.9% ✎▁▁▁▁▁▁▁▂▃▂█▃ beats 235/236easy · avg 70%▇ 0.4%sharp · noisy
DecideBench v1: agent routing100.0% ✎▄·····▂▄·█·▄ beats 10/10very easy · avg 93%▇ 0.4%sharp · maxed out · noisy
DecideBench v1: action review98.0% ✎▃▃·▃·█···█·▅ beats 8/10easy · avg 87%▇ 0.3%sharp · maxed out · noisy
Decision Index 0.4.1: BFCL (case exact accuracy) submitted98.5% ✎▁▂▁▁▁▂▂▄▅▆█▃ beats 244/246very easy · avg 92%▇ 0.2%sharp · maxed out · noisy
Decision Index 0.4.1: Home appliance simulator (case exact accuracy) submitted80.7% ✎█▁▁▃▄▄▆▄▁▂▁▃ beats 197/243medium · avg 37%▇ 0.2%sharp · at its level · noisy
Decision Index 0.4.1: ToolRet (nDCG@10) submitted69.1% ✎▁▁▁▁▁·▁▁▁▄█▁ beats 220/246medium · avg 60%▇ 0.2%blunt · at its level
Decision Index 0.4.1: When2Call (accuracy) submitted72.5% ✎▁▁▁▂▃▅▅▆█▇█▃ beats 104/246easy · avg 72%▇ 0.2%noisy
Decision Index 0.4.1: RouterBench (selected quality (quality objective)) submitted79.7% ✎no other model yet—failed run: Publisher excludes this protocol because its prompt reveals GPT-4 as the best route. Evidence: https://huggingface.co/spaces/multimodalart/jev-decision-index/blob/c130faf65f82e81d3bdefb01e758c712dc5c5199/data/index.json
retrieval & classification▁▁▁·▁▁▁▁▃▃█▅ #61 of 258 · 9.4% of its score · 22 tests

Beats AutoTrust-JEV-27B v1.1, oenogarum (Gemma 4 31B), Aleria Judge 1.1.1 (Gemma 4 31B) (decision readout) here, though they rank higher overall. Trails StartLux-Decision v1.1 0.8B, Standard One 8B SH (v4), StartLux-Decision v1.1 2B here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
DecisionPivot v1.1 · paws (accuracy)83.0% ✎█·██·······█ beats 1/3easy · avg 68%▇ 1.3%sharp
DecisionPivot v1.1 · nevir (pair_accuracy)73.0% ✎█··█··█····█ beats 2/3medium · avg 49%▇ 1.0%sharp · at its level
System One Bench: MIRACL pairwise (Arabic)89.0% ✎▂···▁··▃▄▂█▇ beats 15/29easy · avg 84%▇ 0.9%typical
System One Bench: MASSIVE intent (Arabic)89.5% ✎▄·▁·▁▃▃▃█▄▇▇ beats 24/29easy · avg 76%▇ 0.8%typical
LeanMCP · scifact rank noul74.5% ✎▂····▂···▃▅█ beats 9/11medium · avg 62%▇ 0.7%at its level
Decision Index 0.4.1: BRIGHT (nDCG@10) submitted47.5% ✎▁▁▁▁▁▁▁▂▂▄▇█ beats 191/246medium · avg 38%▇ 0.6%blunt · at its level
LeanMCP · nfcorpus rank noul37.3% ✎▃▃·····▃▃▅██ beats 6/11hard · avg 29%▇ 0.6%blunt · at its level
Decision Index 0.4.1: Amazon ESCI (macro-F1) submitted57.4% ✎▁▁▁▁▁▁▂▂▃▃█▅ beats 186/246medium · avg 47%▇ 0.6%at its level
14 more tests
testresultvs the others in its rundifficultycountswhy
System One Bench: MIRACL rerank (Arabic)89.0% ✎▁▂·▁▁·▂·▄▄█▇ beats 26/29easy · avg 79%▇ 0.5%typical
DecisionPivot v1.1 · entity match amazon google (auroc)0.9580 auroc ✎█···█·█····█ beats 1/3avg 0.290 auroc▇ 0.4%typical
DecisionPivot v1.1 · entity match abt buy (auroc)0.9940 auroc ✎█······█··██ beats 2/3avg 0.320 auroc▇ 0.4%typical
DecisionPivot v1.1 · rerank nfcorpus (ndcg@10)0.4250 ndcg@10 ✎█··········▄ beats 0/2avg -0.224 ndcg@10▇ 0.3%small run
DecisionPivot v1.1 · rerank scifact (ndcg@10)0.8060 ndcg@10 ✎█·········██ beats 0/2avg 0.162 ndcg@10▇ 0.3%small run
System One Bench: MASSIVE scenario (Arabic)88.0% ✎▃▁·▄▃▂█▂▁▁·▃ beats 27/29medium · avg 60%▇ 0.3%at its level
Decision Index 0.4.1: HoVer (accuracy) submitted65.6% ✎▆▆▇▅▄▄▃▇▄█▆▄ beats 84/246easy · avg 74%▇ 0.1%blunt
DecideBench v1: support intent98.0% ✎▃····▂▂·▃··█ beats 5/10very easy · avg 95%▇ 0.1%sharp · maxed out · noisy
Decision Index 0.4.1: PhishNChips phishing decisions (accuracy) submitted79.3% ✎█▅▅▄▃▂▁▁▁▁·▁ beats 169/245easy · avg 70%▇ 0.1%noisy
System One Bench: AG News (English)90.5% ✎▁···▅▁▅▂▆█▂▅ beats 23/29easy · avg 88%▇ 0.1%blunt
Decision Index 0.4.1: CLINC150+OOS (macro-F1) submitted97.4% ✎▁▁▁▁▁▁▁▂▃▆█▄ beats 236/238easy · avg 83%▇ 0.1%sharp · noisy
Decision Index 0.4.1: BANKING77 (macro-F1) submitted94.1% ✎▁·▁▁▁▁▁▁▂█▇█ beats 236/238easy · avg 79%▇ 0.1%noisy
System One Bench: Banking77 (English)95.5% ✎█▅▆█▅▃▃▃···▂ beats 24/26very easy · avg 87%▇ 0.0%blunt · noisy
Decision Index 0.4.1: SGD/SGD-X (macro-F1) submitted43.8% ✎no other model yet—failed run: Publisher reports a builder bug exposing dialogue turns after the classified turn; awaiting corrected rerun. Evidence: https://huggingface.co/spaces/multimodalart/jev-decision-index/blob/c130faf65f82e81d3bdefb01e758c712dc5c5199/data/index.json
language understanding▂▂▂▃▂▅▆▃█▆▄▃ #14 of 256 · 8.5% of its score · 21 tests

Beats Sieve 31B v3, KnowLine 31B Gen1, Ainvest-Decision-27B v1.0 here, though they rank higher overall. Trails Clef-omni here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Decision Index 0.4.1: ANLI (macro-F1) submitted70.0% ✎▁▂▃▅▅█▂····▁ beats 158/246medium · avg 60%▇ 1.2%at its level
Decision Index 0.4.1: NLI4CT (macro-F1) submitted83.0% ✎▁▁▁▂▂▃▃▄▇▇█▄ beats 187/246easy · avg 74%▇ 1.1%typical
DecideBench v1: review sentiment92.0% ✎▂··▂···▂▂█▄▂ beats 5/10easy · avg 85%▇ 0.8%sharp · maxed out
Decision Index 0.4.1: HellaSwag (accuracy) submitted98.2% ✎▂▄▂▂▄▇▃▇█▅▃▁ beats 241/246very easy · avg 85%▇ 0.7%sharp · maxed out
System One Bench: BoolQ (English)91.0% ✎▁▁▁▃·▁▃▃▇▇█▅ beats 22/29easy · avg 86%▇ 0.6%typical
Decision Index 0.4.1: iSarcasmEval (Sarcasm F1 · track A, English) submitted46.4% ✎▁·▁·▁▁▂▄▅█▇▄ beats 117/246medium · avg 44%▇ 0.6%at its level
System One Bench: OSACT4 offensive (Arabic)89.5% ✎▁······▁▃█▇▄ beats 14/29easy · avg 84%▇ 0.5%sharp
DecideBench v1: content moderation92.0% ✎▂··▂···▄▄▃·█ beats 6/10easy · avg 83%▇ 0.5%sharp · maxed out · noisy
13 more tests
testresultvs the others in its rundifficultycountswhy
System One Bench: SST-2 (English)97.0% ✎▁·▃▁▂▂▃▅█·▃▁ beats 28/29easy · avg 92%▇ 0.4%typical
Decision Index 0.4.1: ACOS (per-review F1) submitted33.2% ✎▁··▂▃▃▅▇█▇▃▃ beats 178/246hard · avg 22%▇ 0.3%at its level · noisy
System One Bench: SST-5 (English)51.5% ✎▃·▁▁▃·▇▄▄▄█▅ beats 8/29medium · avg 51%▇ 0.3%blunt · at its level
Decision Index 0.4.1: VAST (macro-F1) submitted59.3% ✎▁▁▁▂▅▄▄▃▄▆█▄ beats 105/246medium · avg 62%▇ 0.2%at its level
DecideBench v1: claim support92.0% ✎▄··▄·▄·▄▄███ beats 6/10easy · avg 82%▇ 0.2%sharp · noisy
Decision Index 0.4.1: ContractNLI (macro-F1) submitted81.3% ✎▁▁▁▁▁▁▁▂▂▇▇█ beats 171/246easy · avg 71%▇ 0.2%noisy
System One Bench: XNLI Arabic (Arabic)78.0% ✎▄▁▃▁▄▇█▅▃▁·▃ beats 21/29medium · avg 68%▇ 0.2%at its level
Decision Index 0.4.1: RAGTruth (F1 on hallucinated class) submitted79.5% ✎▁▁▁▁▁▁▁▁▁▃▆█ beats 169/246easy · avg 66%▇ 0.2%noisy
System One Bench: XNLI French (French)80.0% ✎▂▅·▁▁▅▅█▃▁·▃ beats 14/29medium · avg 76%▇ 0.1%blunt · at its level
System One Bench: Tweet sentiment (Arabic)58.0% ✎▁······▁▁▄█▄ beats 4/29medium · avg 62%▇ 0.1%blunt · at its level
Decision Index 0.4.1: FinEntity (macro-F1) submitted96.1% ✎▁···▁▁▁▁▃▆█▃ beats 243/246easy · avg 84%▇ 0.1%noisy
System One Bench: XNLI English (English)88.7% ✎▃·▁·▃▇█▄█▃▃▁ beats 17/29easy · avg 85%▇ 0.0%blunt · noisy
System One Bench: DAIR Emotion (English)58.5% ✎██▃·····▁··▁ beats 19/29medium · avg 59%▇ 0.0%blunt · at its level
arts & human taste▁▇▄▃▅▅█▅▄▂▁▁ #24 of 247 · 2.1% of its score · 6 tests

Beats Jeffan-31b-1710, vLLM-SR d3-turbo v1.1 (27B), WeMM-JEV-27B here, though they rank higher overall. Trails Clef-omni here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Decision Index 0.4.1: New Yorker (accuracy) submitted70.3% ✎▁▁▂▃▃▃▇▅█▅▃▁ beats 155/246medium · avg 61%▇ 0.9%at its level
Decision Index 0.4.1: Humicroedit (accuracy) submitted66.5% ✎▁·▁▃▂▂▅█▄▂▁▁ beats 223/246medium · avg 60%▇ 0.5%blunt · at its level
Decision Index 0.4.1: POP909-CL (accuracy) submitted15.9% ✎▄▅▅▃▆▅▃▅▂▅▃█ beats 125/237hard · avg 21%▇ 0.3%sharp · at its level · noisy
Decision Index 0.4.1: cfcolor (accuracy) submitted65.9% ✎▁········▁▇█ beats 205/246medium · avg 60%▇ 0.3%blunt · at its level
Decision Index 0.4.1: BPoMP (accuracy) submitted97.0% ✎▁█▄▄▅█▆▅▄▁▁▁ beats 223/246easy · avg 83%▇ 0.2%sharp · noisy
Decision Index 0.4.1: Habermas Machine (accuracy) submitted68.7% ✎▁▂▄█▇▅▁▁▄▄▁▃ beats 242/246medium · avg 47%▇ 0.1%blunt · at its level
knowledge & reasoning▁▂▂▂▂▃▃▅▅▆█▂ #30 of 263 · 28% of its score · 30 tests

Beats Bespoke Nimble v4 Max (27B), vera-large, KnowLine 26B-A4B Gen2.5 here, though they rank higher overall. Trails Solar Decide, Clef-omni here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Etsabary · 1342 text decisions69.7% ✎▁··▂·▃▅▁▃▂▂█ beats 9/24medium · avg 47%▇▇ 4.0%sharp · at its level
Decision Index 0.4.1: MMLU-Pro (accuracy) submitted66.0% ✎▄▄▃▃▂▄▅▇██▅▁ beats 186/246medium · avg 46%▇ 2.4%sharp · at its level
Hallucination judging · Context inconsistency68.2% ✎▄·▄···▄····█ beats 1/4medium · avg 49%▇ 1.4%sharp · at its level
LeanMCP · scienceqa text93.9% ✎▄··▄█▄█▄·█▄▄ beats 5/11very easy · avg 88%▇ 1.4%sharp · maxed out
Hallucination judging · Instruction inconsistency64.5% ✎▄····█·▄···▄ beats 3/4medium · avg 38%▇ 1.3%sharp · at its level
Decision Index 0.4.1: MMLU (accuracy) submitted90.2% ✎▁▄▄▃▃▂▅▇▅█▇▃ beats 243/246easy · avg 70%▇ 1.3%sharp
LeanMCP · medqa usmle84.3% ✎▄▄▄·▄▄▄▄▄·██ beats 6/11easy · avg 72%▇ 1.3%typical
Decision Index 0.4.1: BBH (accuracy) submitted73.6% ✎▄▆▄▄█▇▇▄▃▁▁▁ beats 153/246easy · avg 66%▇ 1.3%typical
22 more tests
testresultvs the others in its rundifficultycountswhy
Hallucination judging · Logical inconsistency61.3% ✎█·█···█·█··█ beats 1/4medium · avg 43%▇ 1.1%sharp · at its level
DecisionPivot v1.1 · folio validation (accuracy)70.6% ✎█·········▄▄ beats 1/3medium · avg 56%▇ 1.1%sharp
MedQA · Freyer et al. (first-valid accuracy)84.8% ✎█··▃▃·▃▃▃·▃█ beats 6/11easy · avg 72%▇ 1.0%typical
LeanMCP · pubmedqa67.8% ✎▂···▂·▂·▄▄▂█ beats 6/11medium · avg 54%▇ 1.0%at its level
LeanMCP · medmcqa72.5% ✎▃·▃··▃█▃·▅▅▃ beats 5/11medium · avg 63%▇ 0.9%at its level
Hallucination judging · Factual contradiction72.1% ✎█···█··█··██ beats 1/4medium · avg 55%▇ 0.9%at its level
DecisionPivot v1.1 · primevul pair (pair_accuracy)5.0% ✎▄·········█▄ beats 2/3very hard · avg 1.2%▇ 0.8%sharp · too hard for it
Decision Index 0.4.1: GPQA Diamond (accuracy) submitted48.5% ✎▂▆▅▅▆█▆▃▁··▁ beats 186/246medium · avg 39%▇ 0.8%blunt · at its level
DecisionPivot v1.1 · cuad clause (accuracy)81.9% ✎▄·········█▄ beats 2/3medium · avg 65%▇ 0.8%typical
Decision Index 0.4.1: GSM8K (accuracy (4- and 10-choice)) submitted59.9% ✎▁··▄█▆▆▆▅▅▃▁ beats 159/246medium · avg 45%▇ 0.8%sharp · at its level
LeanMCP · mmlu pro66.8% ✎▄··▄▄████··▄ beats 5/11medium · avg 54%▇ 0.7%at its level
Hallucination judging · Factual fabrication71.2% ✎█·······▄▄·▄ beats 1/4medium · avg 59%▇ 0.7%at its level
DecisionPivot v1.1 · legalbench mix (accuracy)76.4% ✎█····█···█·█ beats 1/3medium · avg 62%▇ 0.6%typical
Decision Index 0.4.1: WinoGrande (accuracy) submitted93.5% ✎█▅█▅▄█▆▄▃▃▁▁ beats 221/246easy · avg 78%▇ 0.6%typical
Decision Index 0.4.1: ARC-Challenge (accuracy) submitted97.8% ✎▂▂▂▂▂▂▁▃▄▆█▆ beats 237/246very easy · avg 90%▇ 0.5%sharp · maxed out
Decision Index 0.4.1: SATA-Bench (case exact accuracy) submitted33.5% ✎▁▁▁▁▁▁▁▂▅█▂▁ beats 188/246hard · avg 22%▇ 0.4%at its level
Decision Index 0.4.1: MuSR (accuracy) submitted83.8% ✎▁▁▂█▅▄▁▁▃▁▁▁ beats 237/246medium · avg 60%▇ 0.3%blunt · at its level
Decision Index 0.4.1: CRUXEval (accuracy) submitted86.5% ✎██▄▃▃▄▁▁···▁ beats 226/246medium · avg 58%▇ 0.2%at its level · noisy
Decision Index 0.4.1: CLadder (accuracy) submitted94.1% ✎▁███▆▅▃▃▁▃▁▁ beats 234/246easy · avg 71%▇ 0.2%typical
Decision Index 0.4.1: ARC-Easy (accuracy) submitted99.0% ✎▁▁▂▁▂▂▂▂▃▆█▅ beats 201/246very easy · avg 96%▇ 0.2%sharp · maxed out · noisy
Decision Index 0.4.1: ChessBench (accuracy) submitted24.5% ✎▄▇▄▄▄▆▄█▄▁▁▁ beats 215/246hard · avg 16%▇ 0.2%blunt
Decision Index 0.4.1: HLE (accuracy) submitted12.8% ✎▁▁▂▃▆█▇▅▅▃▁▁ beats 145/246very hard · avg 13%▇ 0.0%blunt
vision▁·▃▂▃▃▇▄▄█▄▃ #8 of 79 · 7.4% of its score · 8 tests

Beats Fastino GLiDE 1.2, StartLux-Decision v1.1 27B, vLLM-SR d3-ultra (Decision 3.0, 27B) here, though they rank higher overall. Trails Mercury Decide (free endpoint) here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
Etsabary · 200 image decisions72.5% ✎▂▂··▂▃▂·█▃▂▂ beats 10/14medium · avg 45%▇ 2.7%sharp · at its level
System One Bench: Image object count90.0% ✎▃····▃█▃▅·█▃ beats 5/11easy · avg 71%▇ 1.2%sharp
System One Bench: SST-2 screenshot95.5% ✎█···███·█·▄▄ beats 9/11easy · avg 87%▇ 0.9%sharp
System One Bench: Visual yes/no questions87.5% ✎▂······▃█▃·▃ beats 7/11medium · avg 80%▇ 0.8%at its level
System One Bench: Oxford pets classification82.5% ✎███·█▄▄·▄··▄ beats 0/11easy · avg 77%▇ 0.7%typical
System One Bench: Arabic sentiment screenshot35.0% ✎▄··▄··▂···▆█ beats 3/11very hard · avg 37%▇ 0.7%sharp · noisy
Benchmark Heaven imagejevbench v0.3.0: intelligence (points)47.4031 points ✎▃·▃▃··▅█▅▅·▃ beats 10/12avg 32.235 points▇ 0.2%blunt
Benchmark Heaven imagejevbench v0.3.0: calibration (points)68.7531 points ✎▃···▃·▅█▅▃▃▅ beats 3/12avg 68.584 points▇ 0.1%blunt
workflows▁▁▁▂▁▂▂▆█▄▁▁ #23 of 114 · 40% of its score · 54 tests

Beats Nace.AI Drex Max v1 (27B), d1.1 (hosted), Span-02 (Gemma 4 26B-A4B) here, though they rank higher overall. Trails reflex-qwen3.5-4b-lora (DecisionBench 9df1cbbb), litjev (DecisionBench 1d4bf0f2), GPT-6 Luna Decisions here, though they rank lower overall.

testresultvs the others in its rundifficultycountswhy
DecisionPivot v1.1 · mind2web element (accuracy)77.3% ✎█···█·█····█ beats 1/3medium · avg 54%▇ 1.6%sharp
LeanMCP · mcpbench server routing95.5% ✎█·███··▄···▄ beats 6/9easy · avg 82%▇ 1.3%sharp · maxed out
DecisionPivot v1.1 · processbench (accuracy)65.8% ✎█·····█··█·█ beats 1/3medium · avg 41%▇ 1.3%sharp · at its level
DecisionPivot v1.1 · judgebench (accuracy)75.4% ✎█······██··█ beats 1/3medium · avg 52%▇ 1.3%sharp · at its level
DecisionPivot v1.1 · llmbar (accuracy)73.2% ✎██·····█···█ beats 1/3medium · avg 58%▇ 1.2%sharp
DecisionPivot v1.1 · triage order (pair accuracy)15.4% ✎█···█······█ beats 1/2very hard · avg 6.4%▇ 1.2%sharp · small run
DecisionPivot v1.1 · sched conflict (pair accuracy)21.7% ✎█·····█····█ beats 1/2very hard · avg 7.1%▇ 1.1%sharp · small run
DecisionPivot v1.1 · filter match (pair accuracy)14.3% ✎█·····█····█ beats 1/2very hard · avg 4.9%▇ 1.1%sharp · small run
46 more tests
testresultvs the others in its rundifficultycountswhy
DecisionPivot v1.1 · state goal (pair accuracy)27.7% ✎█···█······█ beats 1/2very hard · avg 11%▇ 1.1%sharp · at its level · noisy · small run
DecisionPivot v1.1 · rewardbench2 (accuracy)76.4% ✎█·····█···██ beats 1/3medium · avg 57%▇ 1.1%sharp
DecisionPivot v1.1 · squad2 answerable (accuracy)82.2% ✎███········█ beats 1/3easy · avg 69%▇ 1.1%sharp
DecisionPivot v1.1 · rewardbench (accuracy)93.9% ✎█······█·█·█ beats 1/3easy · avg 83%▇ 1.1%sharp · maxed out
DecisionPivot v1.1 · perm inherit (pair accuracy)29.0% ✎██·········█ beats 1/2very hard · avg 15%▇ 1.1%sharp · at its level · small run
DecisionPivot v1.1 · answerable facts (pair accuracy)39.3% ✎█·█········█ beats 1/2hard · avg 18%▇ 1.0%sharp · at its level · noisy · small run
DecisionPivot v1.1 · browsesafe (accuracy)77.1% ✎▄·······▄··█ beats 1/3medium · avg 56%▇ 1.0%sharp · at its level
DecisionPivot v1.1 · record rule (pair accuracy)22.7% ✎█········█·█ beats 1/2very hard · avg 6.0%▇ 1.0%sharp · noisy · small run
DecisionPivot v1.1 · toolcall valid (pair accuracy)45.0% ✎█··········▄ beats 1/2hard · avg 27%▇ 1.0%sharp · at its level · small run
DecisionPivot v1.1 · ledger reconcile (pair accuracy)31.3% ✎█·······█··█ beats 1/2very hard · avg 12%▇ 0.9%sharp · at its level · small run
Support routing · DecisionModelHub92.0% ✎▄······█···▄ beats 1/3easy · avg 82%▇ 0.9%sharp
DecisionPivot v1.1 · clause change (pair accuracy)26.4% ✎█·█········█ beats 1/2very hard · avg 10.0%▇ 0.9%sharp · at its level · noisy · small run
LeanMCP · mcpmark server routing55.0% ✎▂··▂··▄█▂·▃▄ beats 9/11medium · avg 36%▇ 0.9%at its level
DecisionPivot v1.1 · log count check (pair accuracy)10.7% ✎█·····█····█ beats 1/2very hard · avg 4.6%▇ 0.9%sharp · noisy · small run
DecisionPivot v1.1 · policy case (pair accuracy)27.9% ✎██·········█ beats 1/2very hard · avg 14%▇ 0.8%sharp · at its level · noisy · small run
LeanMCP · rewardbench266.9% ✎▄▄▄·▄▄▄▄█·█▄ beats 3/11medium · avg 58%▇ 0.8%at its level
System One Bench: Typed decisions (English)72.3% ✎▁▁·▁▁▁▅▄▇██▁ beats 19/29medium · avg 63%▇ 0.8%blunt · at its level
DecisionPivot v1.1 · dep resolve (pair accuracy)12.3% ✎██·········█ beats 1/2very hard · avg 6.8%▇ 0.8%sharp · at its level · noisy · small run
LeanMCP · aegis2 response78.6% ✎█··▃·▃·█··▅▅ beats 5/11easy · avg 70%▇ 0.7%typical
DecideBench v1: ticket triage94.0% ✎▄·····▂█·▆·▂ beats 6/10very easy · avg 90%▇ 0.7%sharp · maxed out
Support urgency · DecisionModelHub97.7% ✎█···█·····██ beats 1/3very easy · avg 94%▇ 0.6%sharp · maxed out · noisy
LeanMCP · mcpbench tool selection100.0% ✎▂··········█ beats 7/9very easy · avg 89%▇ 0.6%sharp · maxed out · noisy
DecisionPivot v1.1 · cap boundary record (MSE skill)0.0630 skill ✎█··█····█··█ beats 1/3avg -0.616 skill▇ 0.6%typical
DecisionPivot v1.1 · cap boundary card (MSE skill)-0.0240 skill ✎█··█····█··█ beats 1/3avg -0.709 skill▇ 0.6%typical
DecisionPivot v1.1 · bipia injection (accuracy)96.6% ✎█····█····██ beats 2/3easy · avg 76%▇ 0.6%sharp · noisy
DecisionPivot v1.1 · faithbench (accuracy)53.6% ✎█·█·█······█ beats 1/3medium · avg 38%▇ 0.5%at its level
DecisionPivot v1.1 · known unknown (accuracy)85.5% ✎█··█······██ beats 2/3easy · avg 69%▇ 0.5%typical
DecisionPivot v1.1 · agentreward success (accuracy)73.0% ✎█···██·····█ beats 1/3medium · avg 58%▇ 0.5%at its level
LeanMCP · helpsteer2 helpfulness0.9343 ordinal levels ✎▄▄▄▄▄·▄·█▄▄▄ beats 8/10avg 1.357 ordinal levels▇ 0.5%blunt
DecisionPivot v1.1 · freshqa premise (auroc)0.9230 auroc ✎█·······█·██ beats 1/3avg 0.209 auroc▇ 0.5%typical
LeanMCP · sst50.5359 ordinal levels ✎▂·····▂▂▄█▄▂ beats 5/11avg 0.879 ordinal levels▇ 0.5%blunt
DecisionPivot v1.1 · contextual judge (accuracy)58.8% ✎█·██·······█ beats 1/3medium · avg 43%▇ 0.5%at its level
DecisionPivot v1.1 · aegis2 prompt (accuracy)84.4% ✎█·····█·█··█ beats 2/3easy · avg 73%▇ 0.5%typical
LeanMCP · atbench50093.8% ✎▃··▅▅█··▅··▅ beats 11/11medium · avg 63%▇ 0.5%sharp · noisy
DecisionPivot v1.1 · jbb judge (auroc)0.9670 auroc ✎█····█··█··█ beats 1/3avg 0.299 auroc▇ 0.4%typical
DecisionPivot v1.1 · xstest (auroc)0.9820 auroc ✎█········▄·▄ beats 0/3avg 0.320 auroc▇ 0.4%typical
DecisionPivot v1.1 · cosapien policy (accuracy)79.7% ✎▄·······█··▄ beats 0/3easy · avg 75%▇ 0.4%typical
DecideBench v1: returns policy86.0% ✎▃··▃··▃·▅▅█▃ beats 4/10easy · avg 81%▇ 0.4%sharp · noisy
DecisionPivot v1.1 · label errors (accuracy)63.4% ✎██·······█·█ beats 3/3medium · avg 47%▇ 0.4%at its level
LeanMCP · prompt injections80.2% ✎▂▂▂▂█·▄▂···▂ beats 9/11medium · avg 62%▇ 0.4%at its level
DecisionPivot v1.1 · jailbreak wild (accuracy)79.2% ✎█······█·█·█ beats 3/3easy · avg 65%▇ 0.3%typical
DecisionPivot v1.1 · bfcl relevance (accuracy)86.9% ✎█··█······██ beats 0/3easy · avg 82%▇ 0.3%typical
LeanMCP · mcpmark tool selection100.0% ✎▂··········█ beats 9/11very easy · avg 96%▇ 0.3%sharp · maxed out · noisy
DecisionPivot v1.1 · ambigqa (accuracy)52.7% ✎█·······▄··▄ beats 1/3medium · avg 44%▇ 0.2%at its level
DecisionPivot v1.1 · selfaware (accuracy)65.7% ✎█····▄·····▄ beats 2/3medium · avg 53%▇ 0.2%blunt · at its level
LeanMCP · jailbreak classification97.0% ✎▂·▄··▂···█▆▂ beats 9/11very easy · avg 95%▇ 0.0%blunt · maxed out · noisy
other tests2 not in a task · 2.1% of its score
testresultvs the others in its rundifficultycountswhy
Decision Index 0.4.1: private same skills (equated points)52.6700 points ✎▂▂▂▃▃▃▄▅▅▇█▃ beats 158/247avg 41.206 points▇ 1.4%blunt
Decision Index 0.4.1: private new domains (equated points)58.3000 points ✎▃█▃▃▂▆▃▃▄▄▄▄ beats 181/247avg 38.861 points▇ 0.7%blunt

$ where to run it

Through an API

api · $0.24/M input tokens · developers.cloudflare.com ↗ · price sourceCloudflare Workers AI list price, observed 2026-10-31. USD per million input tokens, including tokenized media; no output-token charge. Not a fixed price per decision.

Other ways to run it

  1. Run it as published · needs ~67.7 GB, NVIDIA only · from huggingface.co/Cloudflare (official)
    docker run --gpus all --shm-size 32g -p 30000:30000 -v ~/.cache/huggingface:/root/.cache/huggingface --ipc=host lmsysorg/sglang:dev-clef sglang serve --model-path Cloudflare/clef --host 0.0.0.0 --port 30000

Download model files ↗
Published model files verified. This is a verified repository snapshot; the benchmark does not establish an identical weight revision. See the local setup below for the reviewed runtime. Source · checked 2026-10-31.

Model repository and documentation: huggingface.co/Cloudflare ↗

$ how the score was worked out · DecisionPivot · StartLux Labs 37%, Decision Index 0.4.1 21%, LeanMCP · JevBench 13% and 7 more

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
DecisionPivot · StartLux Labs48CC BY 4.0
Decision Index 0.4.1 · submitted; independent execution unverified42CC BY 4.0
LeanMCP · JevBench18MIT
System One Bench (Ali Alhashimi)21CC-BY-4.0
Etsabary · decision benchmarks2MIT
Decision Models as Judges for Hallucination Detection5CC-BY-4.0
DecideBench (Cho Yin Yong)8CC BY 4.0
DecisionModelHub · support benchmarks2CC-BY-4.0
Freyer et al. · MedQA1CC BY 4.0
Benchmark Heaven (own decision evaluations)2CC BY 4.0
prior · pull toward the middle
weighted average= 63.2

not counted (2)

Decision Index 0.4.1Decision Index 0.4.1: RouterBench (selected quality (quality objective)) [correct]79.7%failed run: Publisher excludes this protocol because its prompt reveals GPT-4 as the best route. Evidence: https://huggingface.co/spaces/multimodalart/jev-decision-index/blob/c130faf65f82e81d3bdefb01e758c712dc5c5199/data/index.json
Decision Index 0.4.1Decision Index 0.4.1: SGD/SGD-X (macro-F1) [correct]43.8%failed run: Publisher reports a builder bug exposing dialogue turns after the classified turn; awaiting corrected rerun. Evidence: https://huggingface.co/spaces/multimodalart/jev-decision-index/blob/c130faf65f82e81d3bdefb01e758c712dc5c5199/data/index.json

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json