Online safety regulation of deepfake abuse: a case study on Australia’s eSafety Commissioner

Deepfake abuse or non-consensual synthetic intimate imagery has evolved into a global crisis. The world’s biggest deepfake abuse websites have amassed billions of views. Google directs traffic to these sites. Visa and Mastercard have facilitated the trade of deepfake abuse material. ‘Nudify’ apps have emerged that allow perpetrators to ‘undress’ images of clothed women. With the popularisation of generative artificial intelligence, text prompts can generate deepfake abuse material with ease. As governments around the world move to regulate this issue through criminal and/or civil laws, some have established domestic online safety regulatory agencies to tackle an array of online harms, including deepfake abuse. Through a case study of the world’s first-of-its-kind online safety regulatory agency, the Australian eSafety Commissioner, this article will investigate the agency’s setup, operations, and responses to online harms, particularly deepfake abuse. This article considers whether the eSafety Commissioner makes Australians safer online. The insights from this case study will be critical as Australia and other countries move to reform or develop their own online safety regulatory agencies. The findings will reveal significant problems with the agency’s complaints-and-removal approach to online harms, warranting a reconsideration of this approach to online safety regulation.