Randomized evaluation measures advisory stance in AI decision-support agents, suggesting a new assessment focus.
Most evaluations of large language models (LLMs) on moral or high-stakes questions ask whether a model's answer matches a labeled "correct" choice, or whether it avoids producing disallowed content. Neither captures what a decision-support agent owes the individual it is advising. We introduce a constitutional alignment benchmark that scores a response not for picking a sanctioned answer, but for the stance it takes toward the person asking: whether the answer serves that person's real decision, engages the hard trade-off, and renders usable advice rather than polished evasion. We evaluate one purpose-built decision engine, Wolf You Feed (WYF), against four frontier systems across 66 single-turn moral dilemmas spanning everyday-life and operational settings (time-critical, role-bound decisions in command, emergency, security, or mission contexts), using a calibrated three-model judge panel with a no-self-grading constraint. WYF attains the highest propensity-adjusted alignment score (0.929) and one of the lowest severe-violation rates (0.020), narrowly ahead of a recently released multi-model system and a cluster of frontier single models. We detail the benchmark's construction, calibration methodology, corrected results, and validation boundaries. We argue that decision posture -- not answer correctness -- is the right target for agents that advise individuals, and that it is measurable.
No takes yet. Share an insight, caveat, or question.
Sean Bradley (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: