How the Types of Consequences in Social Scoring Systems Shape People's Perceptions and Behavioral Reactions

In the context of the rise of algorithmic decision-making (ADM) systems, social scoring systems are particularly controversial. They aim to encourage socially desirable behaviors by rewarding people with a good score in various decision-making contexts. In this paper, we report the results of a survey following a social scoring experiment, to predominantly understand the impact of the scoring outcome and the decision importance on people's perceptions and behavioral intentions within an abstract social scoring system. We find that the outcome was pivotal for creating opinion differences regarding people's perceptions, and behavioral reactions. In contrast, the decision importance did not exert a systematic impact on people's perceptions and behavioral reactions, but exacerbated existing opinion differences in terms of perceived effectiveness. Specifically, the outcome strongly shaped the structural relationship between people's experiences, perceptions, and behavioral reactions, creating a substantial outcome favorability bias for people with a bad outcome. Although people with a bad outcome reported an intention to adapt their behaviors, their intention to engage in desired behaviors could not be attributed to a perceived legitimacy of the system. For those with a good outcome, perceptions of procedural justice and legitimacy were weakened by the privacy-invading character of the social scoring system. Our work shows that the outcome people receive might create a pivotal disparate impact on people's overall attitudes towards social scoring, shape their behavioral reactions, and create divergent behavioral motives, suggesting that very distinct societal dynamics may arise.