Benchmarking analysis reveals that doubly robust off-policy evaluation alters optimal model rankings in multi-treatment marketing, indicating the necessity of robust policy assessment.
Multi-treatment uplift modeling estimates the incremental impact of each brand action relative to a baseline, enabling scalable and data-driven personalized brand communication. This paper presents a reproducible benchmarking study of lightweight multi-treatment uplift learners across three marketing datasets covering different brand communication contexts: Hillstrom email creative selection, Bank contact channel allocation, and Starbucks promotional offer personalization. Semi-synthetic scenarios with known oracle policy values are also constructed. Learned policies are evaluated using matched-only estimation and three off-policy estimators, namely IPS, SNIPS, and doubly robust estimation. Evaluation is combined with overlap diagnostics, including clipping rate, effective sample size, and maximum importance weight, as well as sensitivity analysis over clipping and trimming choices. Across real datasets, the doubly robust estimator changes model selection in a non-trivial way. On Hillstrom, matched-only evaluation selects MMOALR, while doubly robust evaluation selects XLearner-LR, showing rank reversal. On Bank and Starbucks, doubly robust and matched-only evaluation agree on the winning model but differ materially in estimated profit levels. The study provides a practical evaluation framework for credible, cost-effective personalized touchpoint allocation and campaign budget optimization.
No takes yet. Share an insight, caveat, or question.
Y. W. Zhu (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: