DynamiGraph: A Specialized, Runtime-Aware FPGA Overlay for Ultra Low-Latency GNN Inference on Edge Devices

Graph Neural Networks (GNNs) have become essential for analyzing graph-structured data, yet their deployment on resource-constrained edge devices is severely limited by high computational complexity and irregular memory access patterns. Here, we introduce DynamiGraph, a specialized FPGA-based overlay accelerator engineered for ultra-low-latency GNN inference in edge computing scenarios. Unlike general-purpose accelerators that incur high resource overhead to support a broad range of operators, DynamiGraph adopts a streamlined architecture focusing exclusively on essential General Matrix Multiplication (GEMM) and Sparse–Dense Matrix Multiplication (SpDMM) kernels. We implement a hardware-native runtime optimization mechanism that dynamically exploits graph sparsity via an edge-centric execution flow, eliminating redundant computations without requiring complex static preprocessing. Experimental results on an AXU2CGA edge platform demonstrate that DynamiGraph achieves sub-millisecond inference latencies on small-scale benchmarks (e.g., Cora) and a peak throughput of 1467 inferences per second. Furthermore, our runtime sparsity exploitation yields over 2000× reductions in floating-point operations compared to dense equivalents. These findings indicate that trading off model generality for architectural specialization and runtime awareness offers an efficient architectural alternative for enabling real-time graph intelligence in power- and bandwidth-limited edge environments.