What question did this study set out to answer?

The aim is to enhance detection methods for malicious code execution in software programs using an innovative algorithm.

February 22, 2026

On the Study of One Way to Detect Anomalous Program Execution

Key Points

The aim is to enhance detection methods for malicious code execution in software programs using an innovative algorithm.
Development of an algorithm based on distance profiles of return addresses in program execution.
Introduction of a new concept called positional distance related to call numbers in a program trace.
Automation tool creation for building distance profiles.
Experimental studies on false detection rates in four well-known browsers.
The proposed algorithm significantly reduces atypical distance detections compared to the basic algorithm with increased verification time.
The effectiveness of the proposed method varies based on the characteristics of the program being protected.

Abstract

Developing more accurate and adaptive methods for detecting malicious code is a critical challenge in the context of constantly evolving cybersecurity threats. This requires constant attention to new vulnerabilities and attack methods, as well as the search for innovative approaches to detecting and preventing cyber threats. This paper examines an algorithm for detecting the execution of malicious code in the process of a protected program. This algorithm is based on a previously proposed approach, when the legitimate execution of a protected program is described by a profile of differences in the return addresses of the called functions, also called a distance profile. A concept is introduced called positional distance, which is determined by the difference between the call numbers in the program trace. The main change was the ability to add to the profile the distances between the return addresses of not only neighboring functions but also several previous ones with the given positional distance. In addition to modifying the detection algorithm, this study develops a tool for automating the construction of a distance profile and experimentally studies the dependence of the probability of false detection of an atypical distance on the training duration for four well-known browsers. The experiments confirm that with a slight increase in verification time, the number of atypical distances detected by the proposed algorithm can be significantly lower than the number of atypical distances detected by the basic algorithm. However, it should be noted that the effect of the transition from the basic algorithm to the proposed one, as the results show, depends on the characteristics of the specific program being protected. The study highlights the importance of continually improving malware detection techniques to adapt them to changing threats and software operating conditions. As a result, this will ensure more reliable protection of information and systems from cyber attacks and other cyber threats.

Bookmark

Cite This Study

Kosolapov et al. (Mon,) studied this question.

synapsesocial.com/papers/699a9ceb482488d673cd2acd https://doi.org/https://doi.org/10.3103/s0146411625700257

Bookmark