Parameter-aware high-fidelity microstructure generation using stable diffusion

Synthesizing realistic microstructure images conditioned on processing parameters is crucial for understanding process-structure relationships in materials design, but it remains challenging due to limited availability of training micrographs and the continuous nature of processing variables. To address these challenges, we present a novel parameter-aware generative modeling approach based on Stable Diffusion 3.5 Large (SD3.5-Large), a state-of-the-art text-to-image diffusion model adapted for microstructure generation. Our method introduces parameter-aware embeddings that encode continuous and categorical variables directly into the model’s conditioning, enabling controlled image generation under specified processing conditions and capturing parameter-driven microstructural variations. To overcome data scarcity and computational constraints, we fine-tune only a small fraction of the model’s weights using DreamBooth and Low-Rank Adaptation (LoRA), efficiently transferring the pre-trained model to the materials domain with minimal new training data. To validate microstructural realism, we have developed a semantic segmentation model based on a fine-tuned U-Net with a VGG-16 encoder on a limited labeled dataset of 24 experimental micrographs. This approach significantly outperforms previous methods in both accuracy (97.1 %) and mean intersection over union (mIoU, 85.7%), ensuring reliable segmentation masks for statistical analyses. Consequently, quantitative analyses using both physical descriptors and spatial statistical functions show strong agreement between synthetic and real microstructures. Particularly, the spatial statistics evaluated via two-point correlation and lineal-path functions yield errors below 2.1% and 0.6%, respectively. These results underscore the novelty and effectiveness of our parameter-aware diffusion approach. To our knowledge, this study marks the first comprehensive adaptation of SD3.5-Large for parameter-aware microstructure generation, providing a scalable and efficient approach applicable to broader scientific domains. These findings highlight the transformative potential of advanced diffusion models to accelerate data-driven materials design.