Answers are grounded in this paper and cite their sources.
A scoring rule is a reward function for eliciting or evaluating forecasts expressed as discrete or continuous probability distributions. A rule is strictly proper if it encourages the forecaster to state his true subjective probabilities, and effective if it is associated with a metric on the set of probability distributions. Recently, the property of effectiveness (which is stronger than strict properness) has been proposed as a desideratum for scoring rules for continuous forecasts, for reasons of “monotonicity” in keeping the forecaster close to his true probabilities, since in practice the forecast must be chosen from a low-dimensional set of “admissible” distributions. It is shown in this paper that what effectiveness implies, beyond strict properness, is not a monotonicity property but a transitivity property, which is difficult to justify behaviorally. The logarithmic scoring rule is shown to violate the transitivity property, and hence is not effective. The L 1 and L ∞ metrics are shown to allow no effective scoring rules. Some potential difficulties in interpreting admissible forecasts are also discussed.
No takes yet. Share an insight, caveat, or question.
Robert F. Nau (1985) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: