Enhanced Malware Detection Using Malware Features: A Comparative Study of LightGBM with Histogram and Byteentropy Analysis

This paper presents a comprehensive approach tomalware detection using malware features from the EMBER2024dataset, focusing on histogram and byteentropy features forenhanced accuracy. We implement and evaluate LightGBM gra-dient boosting complemented by malware binary image visualiza-tion techniques. Our methodology processes large-scale trainingdatasets with focused feature extraction, achieving exceptionalperformance metrics. The LightGBM Win32 model demonstratessuperior performance with F1-scores reaching 0.9832 and AUCscores of 0.9981. We contribute a novel feature extractionpipeline emphasizing histogram byte distribution and byteen-tropy patterns, validated through comprehensive training andcross-validation evaluation. Our results show significant improve-ments over traditional approaches, with particular effectivenessagainst packed and encrypted malware samples. The integrationof binary image visualization provides additional insights intomalware structure, enabling multi-modal detection capabilities.This research advances the state-of-the-art in machine learning-based malware detection through optimized feature engineeringand model architecture design.