The UK’s media regulator Ofcom has launched a formal investigation into the social media platform X, owned by US billionaire Elon Musk, over concerns that its artificial intelligence tool Grok has been used to generate sexualised images of women and children.

The investigation is being carried out under the British Online Safety Act, which places legal obligations on online platforms to prevent users from encountering illegal and harmful content.

Ofcom said it is examining whether X failed to carry out adequate risk assessments relating to Grok’s image-generation capabilities, and whether the company took appropriate steps to prevent the creation and spread of illegal content.

The regulator is also looking at whether effective safeguards were in place to protect children, including age-verification measures.

The probe follows reports that users were able to prompt Grok to create sexualised images, including non-consensual depictions of women and imagery involving children. Some of the material is alleged to fall under the UK’s definition of child sexual abuse material, which is illegal to produce, possess or distribute.

Ofcom said it had contacted X earlier to request information about how it was complying with its legal duties. After reviewing the response, the regulator decided to open a formal investigation.

X has since made changes to Grok’s image tools, including restricting certain features to paying subscribers. However, the move has been criticised by politicians and child protection groups, who argue that limiting access does not address the underlying risks posed by the technology.

A government spokesperson said companies must ensure their platforms are safe by design, adding that monetising access to potentially harmful tools was not an acceptable solution.

If Ofcom finds that X has breached the Online Safety Act, it has the power to impose fines of up to £18m or 10% of a company’s global annual revenue, whichever is higher. In extreme cases, the regulator can also seek court orders to restrict access to a service in the UK.

Elon Musk has previously said that the UK government wanted “any excuse for censorship” on his platform.

Musk, who bought X in 2022, has promoted Grok as a less restrictive alternative to other AI chatbots. X has previously said it is committed to user safety and complies with local laws in the countries where it operates.

Copied