Meta uses employee mouse movements and keystrokes for AI training

Meta uses employee mouse movements and keystrokes for AI training
Mark Zuckerberg, Meta CEO, wears the Meta Ray-Ban Display glasses, as he delivers a speech in Menlo Park, California, U.S., 17 September, 2025
Reuters/Carlos Barria

Meta, which owns Facebook and Instagram, said it's installing software on its employees computers to capture keystrokes and mouse movements to use to train its artificial intelligence (AI) agent models. 

It's part of a broad initiative to build AI agents that can perform work tasks autonomously, the company told staffers in internal memos seen by Reuters.

The company said it will use a tool called Model Capability Initiative (MCI) which is hoped will improve AI's functionality to use dropdown menus and keyboard shortcuts. 

That's according to one of the memos posted by a staff AI research scientist on Tuesday (21 April) in a channel for the company's model-building Meta SuperIntelligence Labs team.

The MCI will run on work related-apps and websites and will take occasional screenshots of employees screens. 

"This is where all Meta employees can help our models get better simply by doing their daily work" one memo said.

Andrew Bosworth, the company's Chief Technology Officer, told employees in another memo shared on Monday, that the company would step up internal data collection as part of Agent Transformation Accelerator (ATA).

“The vision we are building towards is one where our agents primarily do the work and our role is to direct, review and help them improve" Bosworth said. He hopes agents can "automatically see where we felt the need to intervene so they can be better next time".

Bosworth did not explicitly spell out how those agents would be trained, but said Meta would be “rigorous” about “building up data and evals for all the types of interactions we have as we go about our work.” 

Meta Spokesperson Andy Stone acknowledged that the MCI data would be among the inputs.

Stone insisted that the data collected wasn't going to be used for performance checking and would not include "sensitive" content, but didn't elaborate what this referred to.

"If we're building agents to help people complete everyday tasks using computers, our models need real examples of how people actually use them - things like mouse movements, clicking buttons, and navigating dropdown menus," said Stone.

A new team, 'Applied AI', was created last month to help improve the coding capabilities of the internal AI software and the company said it has deployed "strong" engineers to carry out the work. The engineers on the team are responsible for creating new AI models and building and testing new infrastructure.

However, the tech giant could face an issue in Europe, according to Valerio De Stefano, a law professor at York University in Toronto, Canada, who says that monitoring like this is likely prohibited because of its use to monitor staff productivity.

In Italy, using electronic monitoring to track employee productivity is explicitly illegal. Courts in Germany have previously ruled employers can deploy keystroke logging only in exceptional circumstances, such as suspicion of a serious criminal offence.

Cutting back workforce

The memos come on the back of mass redundancies across the company, starting on 20 May. Another phase of job cuts are set to be later this year. Around 8,000 workers are expected to be laid off.

The company's move is among a trend in Silicon Valley where distribution company Amazon confirmed it was cutting 30,000 employees last year and fintech company Block reduced it's staff nearly by half.

Meta has been urging staff to use AI agents for tasks such as coding, regardless of whether these actions slow the worker down. It has also been wiping out distinctions between certain job functions in favour of a new general-purpose job title called “AI builder.”

The tech giant has been moving aggressively to integrate AI into its workflows and reshape its workforce around the technology, arguing it will make the company operate more efficiently.

AI has become common-place and used widely in tech companies, helping to create apps and consuming large quantities of data. Concerns have been raised by trade unions and other groups that replacing staff with AI leads to unemployment, with the United Nations saying women will be disproportionately affected.

Tags