Algorithmic Governance: Gender Bias in AI-Generated Policymaking?

Abstract Artificial Intelligence (AI) tools are becoming deeply embedded in everyday life and increasingly influence or automate decision-making processes that could shape not only public opinion but also policies. As their potential impact grows, it is essential to assess the inclusivity of the policy recommendations they could generate and potential biases they may reinforce. This study examines whether AI systems inherently consider gender in policy proposals, both when gender is explicitly mentioned in prompts and when it is not. We conduct four experiments across diverse policy-making contexts to evaluate whether AI-generated recommendations include, overlook, or misrepresent gender considerations. We tested these experiments in two different AI tools, namely ChatGPT (GPT-4) and Microsoft Copilot. To ensure neutrality and reproducibility, we minimize user-specific context and repeat each prompt multiple times. Our findings offer insights into the limitations of current AI tools as policy advisors and contribute to ongoing discussions on algorithmic fairness, implicit gender bias, and the need for gender-aware AI governance. They also raise broader questions about how AI tools understand and represent gender, and how these representations influence the politics of policy-making.