The Validity of Impressions as a Media Dose Metric in a Tobacco Public Education Campaign Evaluation: Observational Study

Background Evaluation research increasingly needs alternatives to target or gross rating points to comprehensively measure total exposure to modern multichannel public education campaigns that use multiple channels, including TV, radio, digital video, and paid social media, among others. Ratings data typically only capture delivery of broadcast media (TV and radio) and excludes other channels. Studies are needed to validate objective cross-channel metrics such as impressions against self-reported exposure to campaign messages. Objective This study aimed to examine whether higher a volume of total media campaign impressions is predictive of individual-level self-reported campaign exposure in California. Methods We analyzed over 3 years of advertisement impressions from the California Tobacco Prevention Program’s statewide tobacco education campaigns from August 2019 through December 2022. Impressions data varied across designated market areas (DMAs) and across time. These data were merged to individual respondents from 45 waves of panel survey data of Californians aged 18-55 years (N=151,649). Impressions were merged to respondents based on respondents’ DMAs and time of survey completion. We used logistic regression to estimate the odds of respondents’ campaign recall as a function of cumulative and past 3-month impressions delivered to each respondent’s DMA. Results Cumulative impressions were positively and significantly associated with recall of each of the Flavors Hook Kids (odds ratio OR 1.15, P<.001), Dark Balloons and Apartment (OR 1.20, P<.001), We Are Not Profit (OR 1.36, P<.001), Tell Your Story (E-cigarette, or Vaping, product use Associated Lung Injury; OR 1.06, P<.05), and Thrown Away and Little Big Lies (OR 1.05, P<.01) campaigns. Impressions delivered in the past 3 months were associated with recall of the Flavors Hook Kids (OR 1.13, P<.001), Dark Balloons and Apartment (OR 1.08, P<.001), We Are Not Profit (OR 1.14, P<.001), and Thrown Away and Little Big Lies (OR 1.04, P<.001) campaigns. Past 3-month impressions were not significantly associated with Tell Your Story campaign recall. Overall, magnitudes of these associations were greater for cumulative impressions. We visualize recall based on postestimation predicted values from our multivariate logistic regression models. Conclusions Variation in cumulative impressions for California Tobacco Prevention Program’s long-term multichannel tobacco education campaign is predictive of increased self-reported campaign recall, suggesting that impressions may be a valid proxy for potential campaign exposure. The use of impressions for purposes of evaluating public education campaigns may help address current methodological limitations arising from the fragmented nature of modern multichannel media campaigns.