Mining optimized Frequent Itemsets Based on Max-miner: A Distributed Approach

This is the authors' abstract. We don't add key points for this paper.

Association rule mining is a significant instrument in data mining. There are many approaches for extracting frequent rules hidden in data. However, most of them have the problems of too many redundant rules and high computational complexity, especially in massive data mining task. To solve these problems, an optimized frequent itemset mining algorithm for association rules based on max-miner (OFI-AR) is proposed, where the maxminer could mine the initial optimized frequent itemsets. The OFI-AR need not to set a parameter named Confidence and can still extract valuable association rules. At first, transactions are divided into serval sub-transactions based on a distributed computing platform Spark. Then, a proposed sequential frequent itemset mining algorithm are used to extract frequent itemsets on sub-transactions. After that, these itemsets are aggregated and be filtered by a proposed value comparison method, which combine the approach of max-miner and depth-first search. The rest of frequent itemsets are regarded as optimized ones and can generate valuable rules. The experimental results demonstrate that the OFI-AR could find valuable association rules without the set of confidence and efficiently reduce the runtime.