Generation of realistic children's book images based on diffusion models

This paper describes a model for generating realistic images for children's books using diffusion models and explains each step of the proposed model. However, the focus is on reviewing the existing area of research on controlled diffusion models. The focus is on ControlNet, a neural network model for controlling stable diffusion models that allows the addition of controls to guide image generation. In this work, edges and sketches have been used as controls. The edges have been tested using the Canny, Sobel and Prewitt edges. Furthermore, the project aims to design a bank of images extracted from children's books to generate new versions of books with realistic images. The ControlNet pre-training model has been used to create realistic images of children's books, specifically classic storybooks; our tests have been carried out with the books 'The Little Prince' and 'The Seagull and the Penguin'. The research findings bear significant practical implications, affirming that ControlNet is a potent tool for generating visually attractive, diverse, and high-quality images. This reassures illustrators and developers working on children's books about the relevance and applicability of this invaluable resource. In addition, it was concluded that using edges as a control in image generation is better, as edges provide more detail of the original image.