A Novel Approach to Detect Anomaly in Payment Transactions

This paper introduces STAR (Systematic Transaction Anomaly Recognition), a novel multi-algorithm ensemble framework that combines Isolation Forest, One-Class SVM, and Local Outlier Factor through consensus voting to detect payment anomalies without labeled data. STAR applies Isolation Forest, One-Class SVM, Local Outlier Factor (LOF), and a voting-based aggregation method to identify anomalies based on model agreement. The mathematical basis employs density, distance, and isolation-based diagnostic statistics and risk-normalized scoring. Our innovative approach uniquely targets micro-transactions and merchant-level anomalies, detecting 28% small-value and 42% merchant-centric anomalies missed by traditional systems. Based on empirical results from over 50 K transactions, STAR exhibits a high recall rate (99.5%) and precision (98.4%), confirming its scalability, generalizability, and, most importantly, its extensibility towards aiding in fraud detection in financial systems characterized by diverse temporal and behavioral patterns.