What question did this study set out to answer?

The research aims to develop an efficient algorithm for mining multi-dimensional associations in e-commerce user behavior using big data.

April 1, 2026

Research on multi-dimensional association mining algorithm of e-commerce user behavior based on big data analysis

Key Points

The research aims to develop an efficient algorithm for mining multi-dimensional associations in e-commerce user behavior using big data.
Developed the DW-PMARS framework with four core layers: data preprocessing, dynamic weight modeling, parallel mining, and dynamic rule evaluation.
Utilized Spark for data integration and constructed a multi-dimensional hypergraph of user behavior features.
Applied an improved weighted parallel FP-Growth algorithm within the Spark MLlib framework to mine associations.
Implemented real-time updates of weights and rules using Flink stream processing technology, integrating GNN for indirect associations.
DW-PMARS achieved a runtime of 285 seconds with peak memory usage of 12.7GB, surpassing FP-Growth's 1245 seconds and 38.5GB.
The high promotion rule proportion for DW-PMARS was 53.8%, significantly higher than FP-Growth's 18.3%.
In simulated scenarios, DW-PMARS responded quickly to changes, showing improved prediction accuracy compared to the static FP-Growth.

Abstract

Traditional association rule mining algorithms face the dual challenges of exponential increase in algorithm complexity with the increase of dimensions and difficulty in capturing the dynamic evolution characteristics of user behavior by static models when dealing with such high-dimensional sparse data. Therefore, this study proposes a multi-dimensional association mining algorithm framework of e-commerce user behavior named DW-PMARS (Dynamic Weighted Parallel Multi-dimensional Association Rule Mining System), which integrates big data analysis. The framework consists of four core layers: data preprocessing and feature engineering layer, dynamic weight modeling and dimensionality reduction layer, parallel association mining layer, and dynamic rule generation and evaluation layer. In the data preprocessing stage, the Spark platform is used to integrate heterogeneous data and construct a multi-dimensional behavior-feature hypergraph covering product ID, category and behavior type. In the dynamic weight modeling, the quantification and screening of feature importance are realized by combining the static basic weight and the formula assignment of dynamic adjustment weight; In the parallel mining layer, the improved weighted parallel FP-Growth algorithm is used to calculate the support of itemsets with the sum of weights, and the parallel mining is realized in Spark MLlib framework. The dynamic rule layer updates the weights and rules in real time with Flink stream processing technology, combines information entropy to measure and evaluate rules, and introduces attenuation mechanism to eliminate low confidence rules. At the same time, it integrates graph neural network (GNN) to mine indirect associations, and identifies potential strong associations through node embedding vector similarity. The experimental results show that the running time of DW-PMARS is only 285s and the peak memory consumption is 12.7GB, which is significantly higher than that of FP-Growth(1245s, 38.5GB), and the proportion of high promotion rules is 53.8%, far exceeding that of FP-Growth (18.3%). In the simulated promotion scenario, DW-PMARS can respond to the sudden change of behavior pattern in real time, and the prediction accuracy quickly rises after a short fluctuation, while the static FP-Growth accuracy drops sharply.

Demander à l'IA

Bookmark