Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation

In many applications, one can obtain descriptions about the same objects or events from a variety of sources. As a result, this will in-evitably lead to data or information conflicts. One important prob-lem is to identify the true information (i.e., the truths) among con-flicting sources of data. It is intuitive to trust reliable sources more when deriving the truths, but it is usually unknown which one is more reliable a priori. Moreover, each source possesses a vari-ety of properties with different data types. An accurate estimation of source reliability has to be made by modeling multiple prop-erties in a unified model. Existing conflict resolution work either does not conduct source reliability estimation, or models multiple properties separately. In this paper, we propose to resolve conflicts among multiple sources of heterogeneous data types. We model the problem using an optimization framework where truths and source reliability are defined as two sets of unknown variables. The objec-tive is to minimize the overall weighted deviation between the truths and the multi-source observations where each source is weighted by its reliability. Different loss functions can be incorporated into this framework to recognize the characteristics of various data types, and efficient computation approaches are developed. Experiments on real-world weather, stock and flight data as well as simulated multi-source data demonstrate the necessity of jointly modeling dif-ferent data types in the proposed framework1. 1.