Active discount factor elicitation via reward modification

Agent behavior is shaped by latent decision parameters that govern how rewards are interpreted and traded off over time. A key example is the discount factor , which encodes time preference. Mis-specifying the discount factor can confound reward-centric behavioral models (e.g., inverse RL), motivating the need to infer time preference directly from behavior. This paper presents methods for discount factor elicitation in finite-state Markov Decision Processes via policy observations and controlled reward modifications. First, we introduce an algorithm that bounds the set of discount factors consistent with an agent’s observed (near-)optimal policy, and show how observations across heterogeneous reward settings progressively tighten these bounds. Building on this result, we propose an active elicitation framework in which an ego agent strategically adjusts rewards (with fixed dynamics) to refine its estimate of another agent’s discount factor. Through case studies, we demonstrate that active elicitation accelerates interval refinement relative to passive observation and enables targeted exploration in strategic multi-agent settings. Overall, our results establish reward modification as a principled mechanism for eliciting discount factors and improving behavioral modeling, prediction, and control.