Tech giant Google has reported receiving over 250 complaints globally regarding the use of its artificial intelligence (AI) software to generate deepfake terrorism-related content. The disclosure was made to Australia’s eSafety Commission, highlighting growing concerns over AI-generated harmful content.
Additionally, Google received 86 reports warning that its AI tool, Gemini, had been used to create child exploitation material. The findings underscore the increasing risks associated with AI misuse, prompting regulators to push for stricter oversight.
AI Deepfake Abuse: A Growing Concern
The Australian eSafety Commission described Google’s report as a “world-first insight” into how AI is being exploited to create illegal and harmful content. Under Australian law, tech firms are required to periodically submit information on their efforts to minimize harm or risk facing fines.
eSafety Commissioner Julie Inman Grant emphasized the need for stronger safeguards:
“This underscores how critical it is for companies developing AI products to build in and test the efficacy of safeguards to prevent this type of material from being generated.”
Google’s report covered the period from April 2023 to February 2024 and reflects the increasing misuse of AI-powered tools. However, the report does not clarify how many of the complaints were verified.
How Google is Handling the Issue
In response to the child abuse reports, Google used a hatch-matching system, which automatically detects and removes known images of abuse. However, the company did not employ the same detection method to filter out AI-generated terrorist or violent extremist content. This gap in enforcement raises concerns about the effectiveness of current AI moderation tools.
The revelations add to the ongoing debate about AI regulation, with governments worldwide calling for stricter controls to prevent AI from enabling crimes like fraud, deepfake pornography, and terrorism.
Regulators Cracking Down on Tech Companies
Australia has been taking an aggressive stance on AI and online safety. The eSafety Commission has previously fined major tech platforms like Telegram and X (formerly Twitter) for failing to comply with its reporting standards on child abuse and terrorism-related content.
X was recently ordered to pay a fine of A$610,500 ($382,000), which it plans to challenge in court. Telegram also intends to contest its fine.
With AI capabilities advancing rapidly, regulators are pushing companies like Google to strengthen their safeguards and take accountability for how their technology is used.
The rise in AI-generated harmful content is becoming a major global issue. While Google has taken steps to combat child abuse material, its lack of action against deepfake terrorism content raises critical questions. As AI tools become more sophisticated, the need for stricter regulations and proactive safety measures is more urgent than ever.
