Inside a warehouse in San Leandro, California, a robot trainer named Andrew Ceja is playing Jenga. But he’s not just being filmed. A headset from neuroscience startup Zander Labs is measuring his brain waves — tracking mental states like surprise, error, and intent — while he works. This isn’t a lab experiment in human cognition; it’s a trial run by the data-tooling company Encord to manufacture the physical training data that doesn’t exist yet. Encord is one of a small group of startups betting that the next bottleneck for humanoid robotics isn’t cleverer model architecture, but the sheer scarcity of real-world data to feed those models.
Vineeth Velmurugan, Encord’s head of robot learning and a veteran of OpenAI’s robotics lab, calls this the “bleeding edge” of the effort. He puts the scale of the problem bluntly: a useful foundational data set would need to be something like five times the size of YouTube’s entire video corpus. That’s why data creation has become a business, not just a research hurdle. Encord, which originally built tools to help companies annotate and manage data, now runs factories that produce “egocentric” video from headset-wearing workers and teleoperated robots. The company then annotates those data sets with physical descriptions to help large language model-based systems actually understand what a robot is doing.
The work with Zander Labs’ brain-wave sensors is an attempt to push that data quality further. Lucas Gehrke, a Zander neuroscientist, explains that the brain activity measured during a task can clue model builders into when they need to deploy their most computationally intensive thinking. It’s a signal that is completely invisible in standard video. While still a trial, the goal is to run a brain-wave-tagged data set through customer robotics models to see if performance actually improves. If it does, tagging cognitive states like confusion or focused intent could become a standard layer of robotic training data, not a sci-fi gimmick.
Meanwhile, Encord’s warehouse is filled with the unglamorous stock of training domestic and industrial robots: fake flowers, plastic vegetables, and server racks. Pilots use leader-follower rigs to generate data on everything from pouring coffee — “very sloshy,” as observed during a recent visit — to plugging in ethernet cables, a task data center operators are desperate to automate. A separate modality uses forearm sensors to capture electrical muscle signals, reconstructing a 3D picture of hand movements even when a camera’s view is blocked. The entire operation is a bet that the winners in physical AI won’t just have the best algorithms; they’ll own the most exhaustively detailed map of what humans actually do with their hands.
