Showcasing volunteer selection bias: Behavior of opt-in and not-opt-in participants in a time-of-use tariff study

Pilot studies evaluating demand response programs, such as variable-pricing, often face volunteer selection bias, which can lead to an overestimation of policy effectiveness. In this work, we directly investigate and quantify this bias through a real-world time-of-use tariff field study in Switzerland. A dual-recruitment strategy was employed, comparing the electricity consumption of a traditional volunteer (opt-in) group with a group recruited via an opt-out strategy from the pool of initial non-volunteers (not-opt-in). The final sample consisted of 30 opt-in and 30 not-opt-in households, whose behavioral response was compared against a 574-household group of non-participants. During the winter peak-price period, the opt-in group significantly reduced their consumption by an average of 44 W (a 12 % decrease from baseline) relative to non-participants. In stark contrast, the not-opt-in group showed a negligible and statistically insignificant reduction of 2.3 W (0.7 %). These findings empirically showcase how relying on opt-in recruitment, which is the dominant strategy in most pilot studies, can significantly overstate program impacts, highlighting the critical need to account for selection bias when designing pilot programs and assessing the effects of energy policies.