Leveraging Missing Ratings to Improve Online Recommendation Systems

“Product recommendation systems” are backbones of the Internet economy, leveraging customers' prior product ratings to generate subsequent suggestions. A key feature of recommendation data is that few customers rate more than a handful of items. Extant models take missing recommendation rating data to be missing completely at random, implicitly presuming that they lack meaningful patterns or utility in improving ratings accuracy. For the widely studied EachMovie data, the authors find that missing data are strongly nonignorable. Recommendation quality is improved substantially by jointly modeling “selection” and “ratings,” both whether and how an item is rated. Accounting for missing ratings and various sources of heterogeneity offers a rich portrait of which items are rated well, which are rated at all, and how these processes are intertwined, while reducing holdout error by 10% or more. The authors discuss ways to implement the proposed framework within existing recommendation systems and its implications for marketers.