Answers are grounded in this paper and cite their sources.
Balancing advertisements with organic content is a challenge for large-scale recommendation platforms. Ads are central to platform revenue, but jointly serving them with organic content creates trade-offs between monetization and user experience. Many production systems frame these decisions as supervised learning problems that optimize immediate ad clickthrough, estimating immediate outcomes under a historical logging policy while treating decisions as independent. Formulating joint ad and organic ranking as long-horizon optimization over action sequences, under policy-induced state evolution, remains an open research problem. Reinforcement Learning (RL) offers a principled framework for modeling recommendation as sequential decision-making under uncertainty, but at scale it must represent high-dimensional state and action spaces, specify robust reward functions, explore safely, and manage substantial computational cost. This practitioner-oriented survey synthesizes prior RL work on the joint optimization of ads and organic content, distilling design choices, trade-offs, and evaluation practices through a unifying taxonomy that organizes each system by how it formulates the Markov Decision Process (MDP) in terms of state, action, and reward, and learns its policy. We find that industrial systems favor hybrid architectures and offline or constrained RL, such as contextual bandits and counterfactual policy evaluation, that learn from logged data while balancing scalability, safety, and interpretability.
No takes yet. Share an insight, caveat, or question.
Ordorica et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: