San Francisco, February 17, 2025 – OpenAI has terminated a group of Chinese ChatGPT accounts that were reportedly used to edit and debug code for an AI-powered social media surveillance tool.
The company said the accounts, part of an operation it dubbed “Peer Review,” attempted to leverage ChatGPT for generating sales pitches for a program designed to monitor anti-Chinese sentiment across platforms such as X, Facebook, YouTube, and Instagram.
According to OpenAI, the network of accounts operated during mainland Chinese business hours, used Chinese-language prompts, and demonstrated a pattern consistent with manual, rather than automated, use. “The operators used our models to proofread claims that their insights had been sent to Chinese embassies abroad, and to intelligence agents monitoring protests in countries including the United States, Germany, and the United Kingdom,” the company stated in a press release.
Ben Nimmo, a principal investigator with OpenAI, said this is the first instance in which the company has uncovered an AI tool being used in this manner. “Threat actors sometimes give us a glimpse of what they are doing in other parts of the internet because of the way they use our AI models,” Nimmo told The New York Times.
The surveillance tool’s code appears to have been derived from an open-source version of one of Meta’s Llama models. Documents indicate that the group also used ChatGPT to generate an end-of-year performance review, in which it claimed responsibility for writing phishing emails on behalf of clients in China.
OpenAI noted that fully assessing the impact of this activity will require input from multiple stakeholders, including operators of open-source models who can provide further insight into the operation.
In a separate action, OpenAI recently banned an account that used ChatGPT to generate social media posts critical of Cai Xia, a Chinese political scientist and dissident living in exile in the United States. The same group was also implicated in generating Spanish-language articles critical of the United States, which were published by mainstream news organizations in Latin America and attributed to either an individual or a Chinese company.
OpenAI’s actions highlight the ongoing challenges of monitoring and regulating the use of AI tools in politically sensitive contexts, particularly as these technologies become increasingly integrated into global information networks.
Read next
23:00
OpenAI
OpenAI has unveiled a new option called Flex processing, an API service designed to provide more affordable AI model usage in exchange for slower response times and occasional resource unavailability.
18:00
Musk and OpenAI
A jury trial in the lawsuit filed by billionaire Elon Musk against OpenAI will begin in spring 2026, as determined by U.S. District Judge Yvonne Gonzalez Rogers on Friday.
22:00
OpenAI
OpenAI has unveiled significant updates to its Advanced Voice Mode feature for ChatGPT, making its AI voice assistant more personable and responsive to user needs.
18:30
Reliance Industries
In a move that could significantly boost India's artificial intelligence landscape, tech leaders OpenAI and Meta Platforms have engaged in separate discussions with Reliance Industries, according to a report by The Information.
21:30
OpenAI and Google
OpenAI is urging the Trump administration to grant AI companies an exemption that would allow them to train their models on copyrighted material—a move it argues is crucial for maintaining America's leadership in artificial intelligence.
What is your opinion on this topic?
Leave the first comment