Dual-Time-Scale Hierarchical Reinforcement Learning for Message-Budgeted Network Flow Intrusion Detection

Graph-based network intrusion detection can leverage communication context; however, fixed samplers allocate identical capacity to layer–relation neighborhoods with heterogeneous utility. Flat threshold-based controllers additionally exhibit state-dependent action semantics and reward signals confounded by concurrent encoder updates. We propose DHRL-MB, a message-budgeted graph-learning framework that decomposes sampling into two hierarchical control levels. A slow-timescale manager distributes a global integer budget across layer–relation coordinates, whereas a fast-timescale worker selects neighbor–edge instances within each assigned quota using a stochastic sampling-without-replacement policy. Controller rollouts are evaluated against budget-matched references while the graph encoder remains frozen, and encoder optimization begins only after both policies are fixed. This reward-isolated training schedule stabilizes action semantics and disentangles adaptive-sampling gains from representation learning. A relation-aware edge classifier subsequently propagates only the selected messages. Across four public NetFlow benchmarks, DHRL-MB improves macro-F1 over the strongest baseline by up to 3.42 percentage points under random splits and 6.67 percentage points under endpoint-disjoint evaluation, while retaining only 34% of full-neighborhood messages. These results demonstrate that hierarchical allocation improves balanced detection by concentrating limited computation on the most informative relations and graph depths.