Despite the high-quality results of text-to-image generation, stereotypical biases have been spotted in their generated content, compromising the fairness of generative models. In this work, we propose to learn adaptive inclusive tokens to shift the attribute distribution of the final generative outputs. Unlike existing debiasing methods, our approach operates without explicit attribute class specification during inference or prior knowledge of the original bias distribution. Specifically, the core of our method is a lightweight adaptive mapping network , which can customize the inclusive tokens for the concepts to be de-biased, making the tokens generalizable to unseen concepts regardless of their original bias distributions. This is achieved by tuning the adaptive mapping network with a handful of balanced and inclusive samples using an anchor loss. Experimental results demonstrate that our method outperforms previous bias mitigation methods without attribute specification while preserving the alignment between generative results and text descriptions. Moreover, our method achieves comparable performance to models that require specific attribute classes or editing directions for generation. Extensive experiments showcase the effectiveness of our adaptive inclusive tokens in mitigating stereotypical bias in text-to-image generation. The code is publicly available at https://github.com/itsmag11/AITTI .
Hou et al. (Fri,) studied this question.