Artificial intelligence tools that can be used to create nonconsensual sexual images may be banned in the European Union, following outcry after users used Elon Musk’s Grok service to generate thousands of undressed pictures of women and children.
The European Parliament’s civil liberties committee on Wednesday approved the bloc’s draft text of its Artificial Intelligence Omnibus law. That included an amendment to outlaw any AI system that generates realistic images "so as to depict sexually explicit activities or the intimate parts of an identifiable natural person” without their consent.
The ban wouldn’t apply if a company has imposed measures restricting the creation of such deepfakes.
The amendment’s approval puts the parliament in line with European governments, who agreed on a similar ban, making it likely the prohibition will be approved and put into law later this year.
While Europe has a number of rules that make it illegal to produce and share sexual material without the subject’s consent, this amendment is the first to specifically target AI platforms. Lawmakers opted to target AI technology besides its users, due to swift improvements in models’ capabilities to produce convincing likeness of real people.
That shift was epitomised by the way Grok – a chatbot accessible to anyone using Musk’s X social network – was used to generate and publish online vast amounts of sexualized images based on pictures of fully clothed people in January. Parent company xAI Corp. restricted the feature after widespread criticism.
A representative for xAI did not immediately reply to a request for comment.
The prohibition adds to a growing corpus of EU laws targeting child abuse sexual material and nonconsensual intimate image sharing. A 2024 directive on violence against women made it an offense to use AI to produce sexual images without the subject’s consent, and the Digital Services Act also punishes social networks that allow the spread of illegal content, including child abuse material.
If the ban is adopted into EU law, developers of advanced AI capable of generating all sorts of audiovisual material will have to prove they have set up restrictions on the images their models create. It is unclear how the AI systems would verify whether the person portrayed in an image has given their consent or not.
The EU is working to pass the AI Omnibus, a comprehensive regulation, that aims to simplify the EU’s sweeping Artificial Intelligence Act. The omnibus is also expected to delay implementation of parts of the AI Act on high-risk applications, originally scheduled to kick in in August 2026, and now pushed to December 2027 and August 2028. That will allow specialist organisations to draft detailed guidance on how to comply with the EU’s AI law, and give companies more clarity.
Regulators in the EU and the UK are formally investigating X and xAI over the Grok incident in January to establish whether it had breached laws on content moderation and online safety. – Bloomberg
