Exploring the reliability of rating systems in personalized decision-making contexts: From axiomatic basis to practical implications

The rating system (RS) serves as an effective tool for capturing and aggregating individual opinions on alternatives in various operational decision-making scenarios. However, the reliability of RSs has long been contested since different RSs often produce different decision outcomes for the same decision problem. To this end, this study investigates the reliability of RSs in personalized decision-making contexts from a novel axiomatic perspective. First, an axiomatic system comprised five core axioms, i.e., responsiveness, monotonicity, independence of irrelevant alternatives, non-dictatorship, and consistency, is constructed to characterize a rational and ideal RS. Then, an impossibility theorem is developed, indicating that no RS can satisfy all five axioms simultaneously. Next, axiomatic properties of the general RS are analyzed in personalized decision-making contexts, and it is found that the consistency axiom is the most fundamental property but the hardest to realize in practice. Then, several tight consistency bounds are analytically proposed to reveal the intrinsic mechanism underlying unreliable decision outcomes and offer corresponding reliability judgment tools. Finally, comprehensive simulation experiments are designed to extend the analytical findings. The results of this study can provide novel theoretical insights for the reliability analysis of RSs and offer actionable guidance for RS-induced decisions in real-world operational scenarios.