Exploring truthfulness and applications of prediction models: Dynamics between learners and experts

Prediction models are prevalent in the real world; the ability to accurately predict future events has historically been a valuable one. New tools and methods to better understand prediction models are being developed every day. Our primary aim is to gain a deeper understanding between interactions of prediction models, those who construct them, and the experts or forecasters they query. Our goals of this dissertation are three-fold. First, we provide guidelines that experts, that is, predictors of events, can use to be better informed about prediction modeling, such as whether or not to be truthful. We explore experts algorithms, where a learner seeks to accurately predict future events given predictions from experts. For settings where the number of experts grows to infinity, we give results about different settings where experts ought to be truthful. We then also relax the assumption that experts must be fully rational and explore whether experts should be truthful in this setting as well. Second, we provide new tools that organizers and architects of prediction models can use to build better models. To do so, we explore forecasting competitions, where forecasters of events compete to predict future events. Following the important results of Witkowski et al. [83, 84], we generalize their setting where events are independent, to settings where events may be correlated. We show that given their forecasting competition mechanism, called Event-Lotteries Forecasting Competition (ELF), forecasters ought to be truthful in settings of two arbitrarily correlated events. We also show that for any three correlated events, ELF does not properly incentivize forecasters to be truthful. We then introduce a new mechanism called Bayesian Network ELF (BNELF) that properly incentivizes forecasters to be truthful for any setting of arbitrarily correlated events and chooses the most accurate forecaster given enough events. Notably, BNELF relaxes the assumption that forecasters cannot update their beliefs of future events, making our mechanism even more applicable to real world settings. Therefore, we build on previous results in the literature and bridge the gap between prediction models and correlated events. Finally, we provide an application of prediction models in the field of hepatopathology, the study of the diagnosis of diseases in the liver. We introduce a stable and foundational diagnosis prediction model using Voronoi diagrams for clinical care in hepatopathology, providing an environment where doctors can gain new insight on the spread of diseases in the liver, as well as a systematic way to convey these new findings to other medical experts. We strengthen our results with empirical and histological data.