Answers are grounded in this paper and cite their sources.
Text-to-image (T2I) diffusion models such as Stable Diffusion, SDXL, PixArt- α and Kandinsky are increasingly served side by side behind a single interface, which raises the question of whether compromising the conditioning of one model compromises the others. We study this with an embedding-injection attack that perturbs the target’s text conditioning at inference time, and ask which other models the same perturbation reaches. A cross-encoder operator formalises three relationships between a target’s and a friend’s conditioning, namely representational identity, containment and disjointness, and we determine which holds for each model pair by comparing encoder weights and the conditioning produced for a common prompt rather than by encoder family name. On $$200$$ MS-COCO prompts the relationship governs the outcome. A per-prompt regression of the friend’s alignment drop on the target’s gives a slope of $$0.769$$ , with a $$95%$$ interval of $$[0.677, 0.855]$$ , for a friend whose text encoder is identical to the target’s, $$0.168$$ with $$[0.083, 0.249]$$ for one that carries an additional component the perturbation cannot reach, and exactly zero for disjoint friends, every adjacent pair being separated by an interval excluding zero. Against an identical-conditioning friend the selectivity gap is $$-0.365$$ , so the attack damages the friend more often than it spares it. The ordering is reproduced on a diffusion-transformer target, and its two ends are reproduced under two evaluators from outside the CLIP family. An explicit friend-preservation term reduces collateral damage on an identical-conditioning friend at sufficient weight but halves the attack and does not make it selective. We also find that sharing an encoder family is not sharing a representation, and that conditioning on more than one text encoder roughly halves the attack.
No takes yet. Share an insight, caveat, or question.
Kwon et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: