Cluster Relation-Augmented Hierarchical Reinforcement Learning for Sparse Knowledge Graph Reasoning

Knowledge graphs (KGs) organize relational facts in a form that supports machine reasoning across many domains. In decentralized Internet of Things (IoT) environments, such graphs can describe interactions among distributed devices and services. However, privacy constraints, intermittent communication, and incomplete observations can leave the resulting graphs sparse and incomplete. This sparsity weakens path availability and makes missing-fact inference difficult. Multi-hop reasoning can recover unobserved links through explicit reasoning paths, but conventional hierarchical reinforcement learning (HRL) approaches may explore inefficiently when only a few valid paths are available. This paper presents attentiON exploratIon of cluster relatioN (ONION), a framework for sparse KG reasoning. ONION introduces a cluster relation augmentation (CRA) module to add cluster relation cues to the action space, together with a relation-level attention (RLA) mechanism to emphasize relations that are relevant to the query. These components guide hierarchical exploration toward more informative paths. ONION is evaluated on four public sparse-KG benchmarks, and the results show competitive performance with the clearest gains on highly sparse datasets.