Recommender System-Induced Eating Disorder Relapse: Harmful Content and the Challenges of Responsible Recommendation

As users’ social media feeds have become increasingly driven by algorithmically recommended content, there is a need to understand the impact these recommendations have on users. People in recovery from eating disorders (ED) may try to avoid content that features severely underweight bodies or that encourages disordered eating. However, if recommender systems show them this type of content anyway, it may impact their recovery or even lead to relapse. In this study, we take a two-pronged approach to understanding the intersection of recommender systems, eating disorder content, and users in recovery. We performed a content analysis of tweets about recommended eating disorder content and conducted a small-scale study on Pinterest to show that eating disorder content is recommended in response to interaction with posts about eating disorder recovery. We discuss the implications for responsible recommendation and harm prevention.