Skyline-Enhanced Association Rule Mining for Numeric Datasets

Abstract For Association Rule Mining (ARM) analysis, numeric datasets undergo a data discretization stage that inhibits interdependence relating information on the numeric variables involved. We report on the use of the Skyline operator to improve the predictive ability of the ARM output. A new technique code-named SNARM (Skyline Numerical ARM) is proposed, one that ranks the association rules in the ARM output using Skyline levels constructed by combining the ARM conviction interestingness measure with Spearman. A new algorithm code-named PAT (Predictive Ability Tester) is devised and used to compare SNARM’s performance to the performance of the ARM-only approach where rule interestingness is determined solely by the conviction value. Two numeric datasets are used: (a) the 1. 7M Jester jokes dataset, and (b) a collection of 37, 053 student assessment grades on 19 STEM courses over a 15 academic years period. SNARM was measured to consistently outperform ARM-only for both real-life datasets, achieving up to an 88% F1 score improvement, depending on the dataset, and the configuration parameters used. For numeric datasets, the rules’ predictive ability can be improved by combining ARM relating information (causality) with information on the interdependence of the numeric variables involved. This is achieved by using the Skyline operator to combine ARM’s conviction with Spearman.