Answers are grounded in this paper and cite their sources.
Proposed micro-batch training method improves efficiency, reducing memory use in GNN training on single machines.
Graph Neural Networks (GNNs) have become essential for graph-based applications. Existing GNN training frameworks primarily adopt sampling-based mini-batch approach, which samples vertex neighborhoods to construct subgraphs for training iteratively, but faces a memory-accuracy trade-off. To address this, naive micro-batch training with gradients accumulation can be applied, yet introduces severe redundancy from repeated neighbor sampling. Recent micro-batch training methods overcome this by adopting batch-level graph partitioning strategies that partition full-batch subgraph sampled from all target vertices to generate micro-batches. Nevertheless, these approaches struggle to balance partitioning overhead with redundancy. This paper proposes PDGNN, a system designed for efficient micro-batch GNN training on single machine, reducing both training time and memory consumption. First, we propose a degree-pruning-based graph partitioning method that efficiently processes on simplified and sparsified multi-level bipartite graphs by removing high-degree vertices, significantly reducing partitioning overhead. Second, we propose a dependency-aware redundancy matching and elimination technique that maximizes inter-micro-batch redundancy matching, and eliminates redundant computation by embedding caching and reuse while maintaining model accuracy. Experiments on real-world datasets demonstrate that PDGNN substantially improves the efficiency of large-scale GNN training, achieving up to 3.5 × speedup in epoch time, 46% lower peak GPU memory consumption, and 64% reduced peak CPU memory consumption compared to state-of-the-art batch-level training methods.
No takes yet. Share an insight, caveat, or question.
Zhang et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: