A Lightweight Ensemble Model Based on Knowledge Distillation and Distributed Data Parallelism for Predicting User Advertising Return on Investment

Advertising plays a pivotal role in enabling businesses to connect with potential customers and promote their offerings. In today’s digital age, advertising channels such as online display ads, social media promotions, and targeted email campaigns dominate the marketing landscape. Given the substantial investments companies make in these channels, evaluating advertising effectiveness through Return on Investment (ROI)—a metric representing the ratio of net profit to advertising expenditure—becomes crucial. Accurately predicting user advertising ROI aids in optimizing campaign strategies, ensuring resources are allocated effectively. Traditional heuristic and rule-based methods often fail to capture the complex relationships in user data, leading to limited predictive accuracy. Recent advancements in machine learning, particularly deep learning, have significantly improved ROI prediction by uncovering intricate, non-linear patterns in large datasets. However, deep learning models can be computationally intensive and challenging to deploy in resource-constrained environments. To address these limitations, this study proposes a novel lightweight distributed ensemble model that leverages distributed data parallelism (DDP), knowledge distillation, and ensemble learning. The framework trains a large teacher network using DDP, followed by distilling knowledge into a smaller student network, and integrates high-level representations with other machine learning models. The results demonstrate improved prediction accuracy and computational efficiency, making the model suitable for real-time advertising ROI forecasting.