Obfuscated file-less malware detection using integrating memory forensics data with machine learning techniques

Purpose Malware remains one of the most critical Internet security concerns, with recent advancements making it increasingly sophisticated and capable of evading traditional antivirus protection. Modern threats have shifted from file-based to file-less malware, which resides in memory and bypasses conventional detection mechanisms. This study aims to propose an improved approach for detecting file-less malware families and sub-families to support effective incident response. Design/methodology/approach The proposed approach integrates memory forensics data with machine learning techniques, utilizing ensemble soft voting to enhance detection. The method focuses on reducing detection time at each categorization level, including binary classification, family classification and subfamily classification, while maintaining high prediction accuracy. Findings The suggested method achieves 99.9% accuracy for binary classification, 88.86% accuracy for multiclass malware family classification and 77.40% accuracy for multiclass malware subfamily classification. Furthermore, the detection time was significantly reduced: binary classification from 3.359002 to 1.895786 seconds, family classification from 10.190352 to 3.694623 seconds and subfamily classification from 12.114225 to 3.737292 seconds when selected features were applied. These results demonstrate the effectiveness of the proposed strategy in mitigating sophisticated file-less malware. Originality/value This research contributes by presenting a novel integration of memory forensics and machine learning with ensemble soft voting to improve both detection accuracy and speed in file-less malware analysis. The approach offers a practical advancement in protecting systems against advanced malicious programs by addressing the challenges of malware classification at binary, family and subfamily levels.