Correlation-Driven Imputation for Heterogeneous Bibliographic Data: A Framework for Library Availability and Scopus Records

Not all bibliographic or circulation data may be available and make downstream feature construction and prediction more difficult. However, filling the corresponding values in the global data into the remaining missing tiles using global mean, median or mode is a very straightforward and simple procedure, but it can ignore possible relations between fields and can generate values that do not conform to the logic of the source domain. This paper introduces a procedure called Correlation-Driven Imputation (CDI) that aims at being domain aware for two data situations: data self-collected from the academic-library and data from a bibliographic source like Scopus. CDI determines the variables that were actually observed for an incomplete field, finds appropriate reference records and infers unknown continuous variables based on distance weighted aggregation and unknown categorical variables based on weighted voting. Finally, an estimate is subjected to consistency checking with a set of constraints on the domain, such as the date constraints on the circulation records. The method is the first step of a bigger procedure: the subsequent text representation, feature selection and classification. The values reported in the synopsis provided are preliminary values, resulting in RMSE/MAE/NRMSE of 0.118/0.071/0.084 for the Library data and 0.136/0.082/0.097 for Scopus data, respectively, and the results of imputation accuracy of 96.8% and 95.9% for the two data, respectively. These are synopsis-reported results which are subject to verification with logs. The paper outlines all the methodological steps, the datasets used, criteria for evaluating the correctness of the data, limitations, and the aspects that require replication.