SparsePhysNet: Physics-Informed Sparse Attention Transformers for Sample-Efficient Molecular Property Prediction

Predicting molecular properties accurately from limited labeled data remains a central bottleneck in computational drug discovery and materials science. Existing graph neural networks and molecular transformers achieve strong performance on large benchmark datasets but degrade significantly in low-data regimes, precisely where experimental annotation is most expensive. We introduce SparsePhysNet, a molecular property prediction framework that incorporates two novel inductive biases derived from first-principles chemistry: (1) a Physics-Informed Positional Encoding (PIPE) that initializes atom representations from atomic orbital theory, electronegativity, and covalent radii, and (2) a Chemical Valence Sparse Attention (CVSA) mask that restricts self-attention to chemically meaningful atom pairs, reducing complexity from O(n²) to near-linear in typical drug-like molecules. We evaluate SparsePhysNet on four MoleculeNet benchmarks under scaffold split: ESOL (1,128 samples), FreeSolv (642 samples), Lipophilicity (4,200 samples), and BBBP (2,039 samples). Against baseline models including Random Forest on Morgan fingerprints and Graph Isomorphism Networks, SparsePhysNet achieves RMSE of 1.095 ± 0.096 on ESOL, 3.037 ± 0.068 on FreeSolv, and ROC-AUC of 0.880 ± 0.016 on BBBP — outperforming the RF-FP and GIN baselines across all regression benchmarks. An ablation study confirms that both PIPE and CVSA contribute independently to performance, with the physics encoding providing the largest benefit in low-data regimes. Code and model weights are available at https://huggingface.co/sibi-ai