Garment Recycle Training and Conditional Garment-Person Outline Attention Guided Virtual Tryon

Virtual try-on, a significant application in computer vision, aims to seamlessly simulate the appearance of clothing on a person from a single image. We propose a Diffusion based tryon approach, solving virtual tryon as a problem of conditional image inpainting. Our method introduces GarNet and OutlineNet as two learnable Stable Diffusion ControlNet encoders conditioned on the garment and person outline images, enhancing the controllability and realism of the generated try-on. We propose a two-stage garment diffusion recycling training strategy, utilising \(x₀\) -parameterization. We estimate the initial clean image that is conditioned on the maximum noised input and feed the same to the same diffusion model again to estimate total noise. This reduces over-fitting and makes our model more generalised. We also introduce a zero garment-outline conditioning (ZGOC) block along with a Garment-Outline Cross Attention layer to optimize garment draping and ensure global consistency in the try-on results. The ZGOC block provides control and adaptability by prioritizing garment details that are most affected by body shape, ensuring precise garment alignment with the person's outline. Our comprehensive experiments on the VITON-HD and Dresscode dataset demonstrates that our proposed approach achieves state-of-the-art realism and controllability in VITON, marking a significant advancement in virtual fashion experiences and online shopping applications.