Maximum Entropy and Bayesian Approach to Constructing Probability Distributions

Understanding the probability distribution ascribed to a given dataset enables commentary on the characteristics of the underlying population and formulation of prospective inferences. If the dataset is sufficiently large, the law of large numbers or the central limit theorem can be employed to ascertain the probability distribution of the dataset. However, in instances where the sample size is relatively limited, estimating the PDF of the population becomes difficult. Furthermore, determining the PDF that best describes the available data introduces an additional level of complexity to the analysis. Failure to consider this complexity can have significant consequences. This paper addresses this challenge by exploring the use of maximum entropy and Bayesian logical inference. The likelihood function is obtained with maximum entropy, and a priori distributions of means and standard deviations are assigned. The posterior distribution functions are constructed using Bayesian logical inference. The distributions derived theoretically were then applied to the milk yields of seven dairy cows. Subsequently, the performance of the Bayesian maximum entropy method (BME) is compared with that of the existing bootstrap method. The simulation results demonstrate the effectiveness, flexibility, and robustness of the BME method in obtaining the mean and variance distributions of the data when the number of data is small.