Voting advice applications typically require users to answer questionnaires before receiving party or candidate recommendations. As users answer more questions, the recommendations naturally become more accurate. However, when users do not complete the questionnaire, the certainty of these recommendations is unknown. In this work, we develop and present a measure to quantify this certainty by introducing an algorithm that estimates the candidate recommendation accuracy—the overlap between early and final recommendations—after each question. Through simulations based on existing voter data, we find that our algorithm is more accurate than heuristic estimates. Additionally, it can identify stable recommendations—candidates who are likely to be among the final recommendations—with fewer false positives. Furthermore, we conduct a user experiment investigating different ways of communicating recommendation certainty to users. Our results show that users answer more questions when they see a preview of stable recommendations, but quit the questionnaire earlier when we display an artificially high candidate recommendation accuracy estimate. Moreover, we find that users appreciate the interface’s simplicity over its accuracy. We conclude that displaying personalized stable recommendations can spark curiosity towards voting advice applications while providing a robust estimate of recommendation certainty for users who submit incomplete questionnaires.
Bachmann et al. (Wed,) studied this question.