An Autonomous-Exploration Method for Mobile Robots Based on Distributional Soft Actor–Critic

Autonomous exploration requires reliable viewpoint evaluation from partial observations. Existing reinforcement-learning methods commonly approximate each action by using a scalar expected return, which is inadequate in unknown environments with heterogeneous topology. Similar local observations may correspond to different hidden structures, causing the same action to produce divergent information gains and travel costs. This perceptual aliasing makes value estimation uncertain and can propagate biased decisions through temporal-difference learning. To address this problem, we propose a distributional-value learning framework for autonomous exploration. The core idea is to replace scalar action evaluation with return-distribution estimation, where the distribution mean measures long-term utility and the variance characterizes uncertainty induced by hidden topology. Based on this formulation, task-oriented value learning is developed to stabilize target estimation and suppress optimistic viewpoint evaluation under conditions involving uncertain returns. Simulation experiments indicate that the proposed adaptation can reduce redundant motion and improve exploration efficiency over representative baselines in the tested scenarios, while a preliminary real-robot deployment suggests the feasibility of transferring the learned policy to a physical platform.