AlloyAlign: A generalizable self-supervised multimodal alignment framework for alloy tensile strength prediction

Multimodal learning with artificial intelligence enhances alloy property prediction by integrating complementary data. However, existing methods in the alloy field typically rely on simple feature concatenation, failing to capture the relationships across scales that are crucial for accurate characterization. In this study, we propose AlloyAlign, a generalizable multimodal framework that integrates structured heat treatment parameters, unstructured scanning electron microscopy images, and semantic descriptions generated from images to predict tensile strengths. Crucially, AlloyAlign employs self-supervised learning to achieve detailed alignment across modalities, encoding qualitative physical knowledge into a unified representation space to coordinate representations from different semantic levels. Validated on Inconel 718, the complete multimodal setting achieves coefficient of determination values of 0.9313 and 0.8904 for yield strength and ultimate tensile strength, respectively. In addition, the transferability of the learned encoder is assessed across strength objectives and from AZ80 magnesium alloy to Inconel 718. Additional experimental and literature cases provide complementary validation under shifted processing conditions. This work provides a multimodal representation framework for alloy property prediction by aligning multimodal physical information to decode relationships between structure and properties.