Artificial Intelligence Off-Policy Evaluation of Sequential Community Clinic Triage Decisions Under Delayed Referral Feedback: A Healthcare Counterfactual Estimator Validity Analysis

Off-policy evaluation (OPE) promises to estimate the value of sequential triage policies from logged community-clinic data, but referral outcomes arrive late and selectively, so the logging policy and the target policy are confounded by the same unobserved severity that drives both the decision and the eventual label. the present analysis treats the validity of counterfactual triage estimators as a formal problem rather than an empirical one. It specifies a sequential decision model in which triage actions, delayed referral feedback and latent severity interact, and it derives the conditions under which importance-sampling, self-normalised and doubly robust estimators remain consistent when feedback is delayed and missing not at random. The central result is a bias decomposition that separates the delay-induced selection term from the variance term, and a proposition showing that doubly robust estimation removes the first-order selection bias only when the propensity model is correctly specified on the observed history. A replication-based evaluation of the ['Intercept', 'exposure', 'group', 'severity'] estimator was conducted: 600 data sets of N = 500 were drawn from a declared data-generating process (intercept 12.0, exposure 0.42, group -3.1, severity 1.35, error SD 4.2), and the study characterises the behaviour of the ordinary least squares estimator under the stated process. Across the 600 replications the estimated bias was -0.00378 for the intercept and 0.00033 for exposure, with RMSE 0.8621 and 0.015 respectively, and empirical coverage of the nominal 95% intervals was 1.0 for both parameters. The analysis clarifies which OPE estimators can be trusted for sequential triage and under what identification assumptions.