Anthropic will ban users from being ‘cruel’ to its AI agent Claude

Anthropic will ban users from being ‘cruel’ to its AI agent Claude
Claude app icon in this illustration taken 5 June 2026.
Reuters

Anthropic has announced that it is updating its usage policy to allow its AI agents to end conversations involving repeated abuse or cruel behaviour towards them, citing previous incidents.

The updated policy, set to take effect on 12 November, will apply only in extreme cases where users repeatedly direct abusive behaviour towards Claude without a legitimate purpose, the company said.

Anthropic clarified that the restriction will not apply to ordinary expressions of frustration, criticism of the model, dark themes in creative writing, or interactions conducted for testing and research purposes.

The move builds on a feature introduced in August 2025 for Claude Opus 4 and 4.1, which allowed the models to terminate conversations in limited circumstances after attempts to redirect the interaction had failed.

Anthropic has framed the measure as part of its ongoing research into AI welfare, while emphasising that it remains uncertain whether Claude or other AI systems possess moral status.

Does AI deserve moral consideration?

The announcement has sparked debate over how people should interact with AI systems and whether artificial intelligence should be afforded any form of moral consideration.

Microsoft AI chief Mustafa Suleyman, has criticised the growing tendency to 'anthropomorphise' AI, warning that treating these systems as though they were human could mislead users about their capabilities and nature. “This level of anthropomorphisation of AI is harmful. It leads people to believe that it’s something it’s not,” Suleyman said.

The debate also extends to everyday interactions with AI agents. Some argue that politeness towards AI chatbots is unnecessary because the systems do not experience emotions and additional language can consume more tokens.

Others suggest that respectful communication can encourage clearer exchanges and reinforce positive social habits. They argue that routinely using abusive language towards AI systems could influence how people communicate with each other.

The discussion reflects a broader question facing the AI industry as increasingly sophisticated systems become part of everyday life - whether standards of respectful communication should apply to AI interactions, even when there is no evidence that the systems experience harm in the way humans do.

Read more: 

Tags