What question did this study set out to answer?

This research aims to address the limitations of scaling and efficiency in mesh simulations using Graph Neural Networks.

June 4, 2026Open Access

Training transformers for mesh-based simulations

Key Points

This research aims to address the limitations of scaling and efficiency in mesh simulations using Graph Neural Networks.
Proposed a Graph Transformer architecture utilizing the adjacency matrix as an attention mask.
Incorporated augmentations like Dilated Sliding Windows and Global Attention for computational efficiency.
Conducted extensive experiments using complex 3D computational fluid dynamics datasets and trained over 60 models.
The smallest model is 7× faster and 6× smaller than MeshGraphNet, achieving similar performance.
The largest model outperforms previous state-of-the-art by 38.8% on average.
Demonstrated remarkable scalability on meshes with up to 300k nodes and 3 million edges.

Abstract

Abstract Simulating physics using Graph Neural Networks (GNNs) is predominantly driven by message-passing architectures, which face challenges in scaling and efficiency, particularly in handling large, complex meshes. These architectures have inspired numerous enhancements, including multigrid approaches and K -hop aggregation (using neighbours of distance K), yet they often introduce significant complexity and suffer from limited in-depth investigations. In response to these challenges, we propose a novel Graph Transformer architecture that leverages the adjacency matrix as an attention mask. The proposed approach incorporates innovative augmentations, including Dilated Sliding Windows and Global Attention, to extend receptive fields without sacrificing computational efficiency. Through extensive experimentation, we evaluate model size, adjacency matrix augmentations, positional encoding and K -hop configurations using challenging 3D computational fluid dynamics (CFD) datasets. We also train over 60 models to find a scaling law between training FLOPs and parameters. The introduced models demonstrate remarkable scalability, performing on meshes with up to 300k nodes and 3 million edges. Notably, the smallest model achieves parity with MeshGraphNet while being 7 faster and 6 smaller. The largest model surpasses the previous state-of-the-art by 38. 8% on average and outperforms MeshGraphNet by 52% on the all-rollout RMSE, while having a similar training speed. Code and datasets are available. https: //github. com/DonsetPG/graph-physics

Mark Helpful

Bookmark

Relay

View Full Paper