Jeeves. Reasoning improves Jev-like decision models

Περίληψη

A reasoning Jev-style classifier with a diffusion drafter, trained with SFT and CISPO.

Acknowledgements Inspired by Kev. Highlights

A 9B Jev-like model (Qwen3.5-9B, LoRA, pointer head) that thinks before it decides, with a block-4 diffusion drafter and the full training code and train/dev/test data. Beats Kev-9B and Jev on test data it was never trained on (0.889 vs 0.822 and 0.857) and on JevBench's public tiers (0.935 vs 0.866 for Jev). Supports yes/no (noul), multiple-choice (choice), and rating (score) questions in the same request, through a Jev-compatible API. About 0.3 s per request without thinking and a 3.3 s median with it on one H100. Can be sped up by truncating chain length. Runs on CUDA (Hopper for the FP8 kernel).

Problem Jev-like models give calibrated decision probabilities, but at low accuracy. A lot of pipelines therefore rely on a reasoning model as a fallback. Jeeves trains a Jev-like Qwen3.5-9B (LoRA and a pointer head) using CISPO to reason before it de...