Decision Models as L2 Rerankers for Content-Based Recommendation: A Pre-Registered Evaluation

In content-based recommendation, a second-stage (L2) reranker decides which retrieved items fill the first screen. Decision models, which answer a typed question about a text with a probability, are an alternative to trained cross-encoders, but whether they separate exact matches from close substitutes is unknown. We cast a decision model as a pointwise reranker that asks whether each candidate exactly matches the request. Under a pre-registered protocol, a gain must hold on both retrieved and fully labelled lists. On Amazon ESCI (1000 requests) and Wayfair WANDS (299), the commercial Jev model, used zero-shot, improves P@10 over embedding order by 0.046 to 0.054 and beats the 0.6B Qwen3-Reranker by 0.02 to 0.03 on ESCI, but not significantly on WANDS. Larger trained cross-encoders (Qwen3-Reranker-4B and 8B, bge-reranker-v2-m3) erase most of this lead: Jev is ahead only on fully labelled ESCI lists. A prompted Qwen3-8B stays behind Jev everywhere. The open decision model CLM-v0.1-8B ranks at random level when scoring a whole list but matches embedding order one product at a time. Jev costs about as much as the 4B model and 13 times bge-reranker-v2-m3. A decision model can serve as an L2 reranker but not demonstrably better than trained cross-encoders.