Enhancing Fashion Retrieval with Constraint Verification

Composed Image Retrieval (CIR) aims to search a target database for images that best align with a user’s intent, conditioned on a reference image paired with modification requirements. Existing CIR architectures typically treat the visual reference and the textual modification as symmetrical inputs, fusing their features into a shared latent embedding space. From a user-centric perspective, however, these multi-modal inputs serve fundamentally asymmetric roles: the reference image acts as a soft semantic anchor, whereas the modification text functions as an explicit requirement specifying precise visual changes. Because current models optimize composition predominantly at a global representation level, these non-negotiable logical constraints are frequently violated during inference, leading to retrieval results that fail to satisfy the user’s explicit instructions. To mitigate this limitation, we introduce a novel, training-free verification framework for fashion retrieval that enforces textual constraint adherence without sacrificing the expressive flexibility of open-vocabulary natural language. Our approach leverages schema-conditioned large language models to extract explicit, structured logical constraints from raw queries during post-processing. A downstream vision-language agent subsequently verifies these constraints against the top retrieved candidate pool to penalize non-compliant images and optimize candidate ordering. Extensive evaluations across standard fashion benchmarks demonstrate that our plug-in framework consistently and significantly enhances the recall metrics of state-of-the-art supervised and zero-shot CIR baselines.