GeoFormer: Geometry-Aware Self-Attention for Modeling Quantum Interactions

GeoFormer is a geometry-aware transformer architecture designed for molecular machine learning and quantum interaction modeling. Traditional approaches such as message-passing neural networks rely on predefined local interactions, limiting their ability to capture complex geometry-dependent atomic relationships. GeoFormer introduces geometry-aware self-attention to dynamically learn atom–atom interactions while incorporating 3D molecular structures. Evaluated on the revised MD17 benchmark, GeoFormer becomes increasingly competitive with SchNet as training data scales, achieving improved atomic force prediction across multiple molecular systems. This project explores the potential of transformer-based architectures for accelerating quantum chemistry simulations and molecular discovery.