San Francisco, February 17, 2025 – OpenAI has terminated a group of Chinese ChatGPT accounts that were reportedly used to edit and debug code for an AI-powered social media surveillance tool.
The company said the accounts, part of an operation it dubbed “Peer Review,” attempted to leverage ChatGPT for generating sales pitches for a program designed to monitor anti-Chinese sentiment across platforms such as X, Facebook, YouTube, and Instagram.
According to OpenAI, the network of accounts operated during mainland Chinese business hours, used Chinese-language prompts, and demonstrated a pattern consistent with manual, rather than automated, use. “The operators used our models to proofread claims that their insights had been sent to Chinese embassies abroad, and to intelligence agents monitoring protests in countries including the United States, Germany, and the United Kingdom,” the company stated in a press release.
Ben Nimmo, a principal investigator with OpenAI, said this is the first instance in which the company has uncovered an AI tool being used in this manner. “Threat actors sometimes give us a glimpse of what they are doing in other parts of the internet because of the way they use our AI models,” Nimmo told The New York Times.
The surveillance tool’s code appears to have been derived from an open-source version of one of Meta’s Llama models. Documents indicate that the group also used ChatGPT to generate an end-of-year performance review, in which it claimed responsibility for writing phishing emails on behalf of clients in China.
OpenAI noted that fully assessing the impact of this activity will require input from multiple stakeholders, including operators of open-source models who can provide further insight into the operation.
In a separate action, OpenAI recently banned an account that used ChatGPT to generate social media posts critical of Cai Xia, a Chinese political scientist and dissident living in exile in the United States. The same group was also implicated in generating Spanish-language articles critical of the United States, which were published by mainstream news organizations in Latin America and attributed to either an individual or a Chinese company.
OpenAI’s actions highlight the ongoing challenges of monitoring and regulating the use of AI tools in politically sensitive contexts, particularly as these technologies become increasingly integrated into global information networks.
Read next
17:30
OpenAI
San Francisco, CA, February 17, 2025 – OpenAI announced on Friday that its AI agent, Operator, is now available for ChatGPT Pro subscribers in a broader range of countries, including Australia, Brazil, Canada, India, Japan, Singapore, South Korea, and the United Kingdom, among others.
20:00
OpenAI
San Francisco, CA, February 18, 2025 – OpenAI is evaluating a proposal to grant special voting rights to its non-profit board as a measure to safeguard its decision-making power amid recent hostile takeover attempts.
19:00
AI startup
Feb 18, 2025 – In a significant move in the competitive AI landscape, former OpenAI chief technology officer Mira Murati has launched Thinking Machines Lab, an AI startup assembling a team of about 30 leading researchers and engineers.
10:50
OpenAI plans to finalise its first custom AI chip, aiming for production by 2026, reducing reliance on Nvidia.
07:18
OpenAI
Elon Musk and a group of investors have submitted a $97.4 billion bid to acquire OpenAI, the company behind ChatGPT. The offer was presented to OpenAI’s board on Monday, according to the Wall Street Journal.
What is your opinion on this topic?
Leave the first comment