Performance and Accuracy Tradeoffs for Training Graph Neural Networks on ReRAM-Based Architectures

Graph neural network (GNN) is a variant of deep neural networks (DNNs) operating on graphs. However, GNNs are more complex compared with DNNs as they simultaneously exhibit attributes of both DNN and graph computations. In this work, we propose a ReRAM-based 3-D manycore processing-in-memory architecture called ReMaGN, tailored for on-chip training of GNNs. ReMaGN implements GNN training using reduced-precision representation to make the computation faster and reduce the load on the communication backbone. However, reduced precision can potentially compromise the accuracy of training. Hence, we undertake a study of performance and accuracy tradeoffs in such architectures. We demonstrate that ReMaGN outperforms conventional GPUs by up to <tex-math notation="LaTeX">9.5× </tex-math> (on average <tex-math notation="LaTeX">7.1× </tex-math> ) in terms of execution time, while being up to <tex-math notation="LaTeX">42× </tex-math> (on average <tex-math notation="LaTeX">33.5× </tex-math> ) more energy efficient without sacrificing accuracy.