Hybrid AI Model for Online Payment Fraud Detection Using Machine Learning

The rapid growth of digital payments has been accompanied by a sharp rise in online payment fraud, creating serious financial and trust-related risks for users, merchants, and financial institutions. Traditional rule-based fraud detection systems depend on predefined thresholds and manually maintained rules, and therefore struggle to keep pace with evolving fraud patterns, producing high false-positive rates and delayed detection. This paper presents a Hybrid AI Model for Online Payment Fraud Detection that combines multiple machine learning classifiers within a single decision framework. The proposed system integrates Support Vector Machine (SVM), Random Forest, and XGBoost classifiers, whose outputs are fused by a soft-voting ensemble to produce a more balanced and reliable final prediction. The severe class imbalance inherent in transaction data is handled using the Synthetic Minority Over-sampling Technique (SMOTE), and numerical features are standardized before training. The framework is further designed to accommodate unsupervised anomaly detection (Isolation Forest and Autoencoders) and expert-defined business rules for detecting novel and known fraud types respectively. The system is implemented as a full-stack web application using a Flask backend, a MySQL database, and an HTML, CSS, JavaScript, and Bootstrap frontend, providing user registration, login, transaction submission, real-time fraud prediction with confidence score, and prediction history. Models were trained on a publicly available transaction dataset of more than six million records and evaluated using accuracy, precision, recall, F1-score, and confusion matrices. The individual models achieved accuracies between 99.87% and 99.93%, and the voting classifier achieved 99.906% on the evaluation subset. The system offers a lightweight, practical, and scalable alternative to computationally heavy deep-learning approaches for secure digital payment environments.