This paper addresses the problem of delta-hedging in illiquid markets, where transaction costs, limited depth of the limit order book, and market impact of large trades play a significant role. Classical approaches based on the Black–Scholes model assume continuous trading and infinite liquidity, which leads to significant distortions in practice. To overcome these limitations, we propose a reinforcement learning approach with a risk-averse Bellman operator. As a training environment, we employ an agent-based exchange simulator with support for trading the underlying asset and options, which reproduces the market microstructure and limit order book dynamics. A DeepLOB convolutional encoder is used to extract order book features and capture hidden liquidity characteristics. Numerical experiments show that the proposed method produces a realized PnL distribution centered around zero with lighter tails compared to the classical Black–Scholes delta-hedger. Furthermore, the risk aversion parameter enables control over the trade-off between mean profitability and tail risk management. The results demonstrate the efficiency of the approach and its applicability for constructing robust hedging strategies in illiquid markets.
Dineev et al. (Wed,) studied this question.