Multimodal deep learning-based image design knowledge pushing and cascade generation method

Image design knowledge pushing and cascade generation face challenges such as insufficient precision in design knowledge pushing and poor quality in image generation. Therefore, this study constructs an image design knowledge pushing and cascade generation model. This model introduces Vision Transformer and Bidirectional Encoder Representations from Transformers to align multimodal features across images, sketches, and texts. Meanwhile, this study utilizes Bidirectional Long Short-Term Memory and Conditional Random Field to construct a design knowledge graph, and uses Light Graph Convolution Network to achieve knowledge pushing. In addition, this study designs a two-stage cascade latent diffusion generation framework, which combines knowledge-guided sampling to achieve controllable image generation. Results indicate that the model converges rapidly and exhibits good fitting performance. In the knowledge pushing task, the Top-1 pushing precision of the model reaches 96.81%, the normalized discounted cumulative gain is 94.63%, and the recall rate is 92.47%. Furthermore, in the portrait photography scenario, the Fréchet Inception Distance of the model is 12.37. In the landscape and architecture scenario, these metrics are 13.96 and 0.199, respectively. The structural similarity index of the model reaches 0.866, and the Peak Signal-to-Noise Ratio is 28.28 dB. Overall, the proposed model possesses high-precision knowledge pushing and high-quality cascade generation capabilities, which provides a technical reference for intelligent image design.