Exchangeability, Correlation, and Bayes' Effect

Summary We examine the difference between Bayesian and frequentist statistics in making statements about the relationship between observable values. We show how standard models under both paradigms can be based on an assumption of exchangeability and we derive useful covariance and correlation results for values from an exchangeable sequence. We find that such values are never negatively correlated, and are generally positively correlated under the models used in Bayesian statistics. We discuss the significance of this result as well as a phenomenon which often follows from the differing methodologies and practical applications of these paradigms – a phenomenon we call Bayes' effect.