The generative creation in new media art under diffusion and ControlNet

This study addresses several limitations of traditional diffusion models in new media art creation, including insufficient control precision, conflicts between semantic and visual conditions, and poor controllability. The study proposes an adaptive two-stage control method that combines large language models with an optimized ControlNet network, aiming to achieve structured guidance and precise control over the generative process. The proposed method comprises three components: structured layout generation, a dynamic conditional weight mechanism, and a Conditional Scaling Predictor. The first component ensures accurate interpretation of natural language prompts, while the second improves coordination among multimodal conditions. Evaluation on the Fill50K dataset demonstrates that the proposed model achieves an Fréchet Inception Distance (FID) of 21.4 and a Structural Similarity Index Measure (SSIM) of 0.869, outperforming baseline models such as the traditional ControlNet and Text-to-Image Adapter (T2I-Adapter) in both generation quality and control precision. User studies further indicate that the method achieves an average score of 4.4 out of 5 in terms of structural fidelity and detail richness. These results suggest that the proposed two-stage control framework can effectively enhance both structural accuracy and artistic expressiveness, providing an innovative solution for new media art creation. Moreover, the proposed method demonstrated strong generalization in cross-style transfer tasks, effectively adapting to the expressive requirements of different artistic genres and providing a technical foundation for art creation. In the future, this framework could be integrated with interactive creative tools to further lower the technical barriers to artistic production.