From deepfakes to grooming: UN warns of escalating AI threats to children

The staggering amount of harmful AI-generated online content has prompted an urgent call from across the UN system for a raft of measures to protect children from abuse, exploitation and mental trauma. Cosmas Zavazava, Director of the Telecommunication Development Bureau at the International Telecommunications Union (ITU) – one of the key agencies that drafted the statement, which includes guidelines and recommendations – catalogues a dizzying array of ways that children are targeted.

This extends from grooming to deepfakes, the embedding of harmful features, cyberbullying and inappropriate content: “We saw that, during the COVID-19 pandemic, many children, particularly girls and young women, were abused online and, in many cases, that translated to physical harm,” he says.

Organisations that advocate for children report that predators can use AI to analyse a child’s online behaviour, emotional state, and interests to tailor their grooming strategy. AI is also enabling offenders to generate explicit fake images of real children, driving a new form of sexual extortion.

The Childlight Global Child Safety Institute, an independent global institute established to gather the most reliable data available on child sexual exploitation and abuse, found in a 2025 report that technology‑facilitated child abuse cases in the US increased from 4,700 in 2023 to more than 67,000 in 2024.

Four young adults are seated on a vibrant pink couch, engrossed in their smartphones. The scene highlights youth engagement with digital technology and social media, reflecting themes of internet safety, cyberbullying prevention, and digital citizenship.

UN Member States are taking stronger measures, as they learn about the scale and severity of the problem. At the end of 2025, Australia became the first nation in the world to ban social-media accounts for children under 16, on the basis that the risks from the content they share far outweighs the potential benefits. The Government there cited a report it had commissioned, which showed that almost two-thirds of children aged between 10 and 15 had viewed hateful, violent or distressing content and more than half had been cyberbullied. Most of this content was seen on social media platforms. Several other countries, including Malaysia, the UK, France and Canada, look set to follow Australia’s lead, preparing regulations and laws for similar bans or restrictions.

Read more>>

Source: The UN