Are Brain Waves the Next Unlock for Physical AI?

The Jenga Game That Could Change Everything

The frontier of physical AI is a Jenga game in a warehouse in San Leandro, California.

That warehouse is occupied by Encord, a company that builds data tooling used to train AI models. Andrew Ceja is a pilot the company's term for its robotic trainers and he's carefully pulling wooden blocks from a tottering tower while wearing a headset with a camera that tracks what he sees. That alone is fairly common for collecting robot training data, but this headset includes sensors that measure his brain waves as he carefully disassembles the block tower.

It looks like a simple game. But what's happening beneath the surface is anything but. Ceja's brain activity the subtle electrical signals that fire as he assesses each block's stability, plans his next move, and reacts when the tower wobbles is being captured and encoded. That data, combined with the visual record of his actions, could become the foundation for teaching robots how to navigate the physical world with human-like intuition.

Encord is one of a small but growing number of startups betting that the next real constraint on humanoid and warehouse robotics won't be model architecture but instead the sheer scarcity of real-world physical training data. Rather than just helping robotics companies manage the data they have, Encord is building a business around manufacturing the data they don't.

The Data Bottleneck

The bet that generative AI can do for robots what it's done for chatbots keeps running into the same wall. Large language models were built on the text of the entire internet billions of words, trillions of parameters. Finding the same raw materials to teach neural networks about physical manipulation is a different challenge entirely.

Self-driving car companies collect their own data, but that's hard to scale. Training from video can work, but it lacks the fidelity of real-world data. Vineeth Velmurugan, Encord's head of robot learning, estimates that it will take a data set something like five times the size of YouTube's video corpus to break through a scale that helps explain why data-generation itself has become a business and not just a research problem.

"The data simply does not exist," Velmurugan said.

This is the fundamental challenge facing physical AI. While chatbots could be trained on the accumulated text of human civilization, robots need to understand the messy, unpredictable, physical world. They need to know how much force to apply when gripping a fragile object. They need to understand how objects behave when they're stacked, moved, or dropped. They need to navigate environments that change from moment to moment.

And that data has to be manufactured, not just collected.

The Brain Wave Experiment

The brain wave headset Ceja is wearing was built by Zander Labs, a German neuroscience startup that's betting measuring brain activity to deduce mental states like error, intent, and surprise can create a more useful data set to train models. Encord's work with Zander is currently a trial run; the goal is to build an initial brain wave-tagged data set, run it through customer robotics models, and evaluate whether it actually improves performance before deciding whether to scale it up.

Lucas Gehrke, a Zander neuroscientist supervising the work, says that the amount of brain activity used at any point during a given task offers clues for model builders trying to figure out when they need to deploy their highest-effort models.

Think about what happens when you're performing a delicate task. When you're threading a needle, your brain is firing on all cylinders. When you're doing something routine, like picking up a coffee cup o opening a door your brain is running on autopilot. By capturing these neural signatures, the researchers hope to teach robots when to deploy their most sophisticated processing and when they can rely on simpler heuristics.

This is the "bleeding edge" of the effort to solve the robotics data bottleneck, according to Velmurugan. A veteran of OpenAI's robot lab and Berkshire Grey, the warehouse automation firm, Velmurugan joined Encord to build the company's internal data-creation team.

Manufacturing Physical Data

Companies building robot brains are now turning to two main sources. The first is "egocentric" video collected by workers wearing cameras, often augmented with additional camera angles and other metrics. The second is collecting data from robots operated remotely. Encord does both, drawing egocentric data from several factories around the globe and using its San Leandro facility to experiment with new modalities, like brain waves, or collect data sets around specific skills for fine-tuning.

When TechCrunch visited, pilots were using leader-follower rigs paired robotic arms, one controlled directly by a human operator and one that mimics its movements to create data about tasks like pouring coffee from a pot into mugs and stacking poker chips. The work is painstaking, requiring precise movements and careful attention to detail.

"Every humanoid company has asked us for these pieces," Velmurugan says.

Storage racks held cartons of fake flowers in vases, books, plastic vegetables, kitty litter trays and scoops, bags and bundles of wires—the stock in trade for training manipulators for household tasks. At one of these stations, another pilot, Sofia Infante, maneuvers robotic arms to plug and unplug ethernet cables from the back of a server the kind of work data center operators would love to be automated, if only robots could manipulate them with the required precision.

Taking a spin behind the controls, it's immediately clear why that's still out of reach. Pincers are far less dextrous than human fingers and lack the degrees of freedom we take for granted in our arms. The gap between what humans can do and what robots can currently achieve is vast, and filling that gap requires vast amounts of training data.

Beyond Vision: New Sensing Modalities

Another new data modality that Encord is developing uses a set of sensors strapped to the forearm to detect electrical signals in muscles. Video taken of human hands manipulating objects typically doesn't capture the entire hand, but Velmurugan hopes to build a 3D depiction of where the hand is at any time based on the arm sensors, creating a more robust understanding for models.

This multimodal approach combining vision, brain activity, muscle signals, and physical metrics could create a much richer dataset than any single source could provide. The goal is to capture not just what humans do, but how they do it, and even why they do it that way.

Encord's data sets are annotated with physical descriptions of what each video contains "right hand tightens bolt," to aid LLM-based models in understanding what is happening. Velmurugan estimates this kind of dense annotation is worth 100 times as much as "junky ego data" for training specific tasks, and it only costs 20 times more to produce a good trade, on paper.

But "20 times more" is still real money, and that's the catch. Scraping text off the internet, the way LLM makers built their models by pulling from Stack Overflow and the rest of the web, cost frontier labs next to nothing. Generating physical training data does not, and that's the limit of the physical-AI-as-LLM comparison. This kind of data has to be manufactured, not just collected, and that changes the economics of building these models.

The Economics of Physical AI

The cost of generating physical training data is the invisible constraint on the robotics industry. While the cost of compute continues to drop and model architectures continue to improve, the cost of creating high-quality physical training data remains stubbornly high.

This is why Encord's business model is so interesting. By positioning itself as a data manufacturer rather than just a data management tool, the company is addressing a fundamental bottleneck in the industry. It's building the raw materials that robotics companies need to train their models, and it's experimenting with new modalities like brain waves and muscle sensors that could make that training more efficient.

Velmurugan says that progress is being made with Encord's visibility into programs across the industry, he's able to see start-ups and frontier labs alike figure out what works and what doesn't to improve physical AI models. That vantage point sitting between many robotics companies at once is also part of Encord's pitch. It can spot which data techniques are gaining traction industry-wide before any single customer can.

That will keep the dozen or so pilots at Encord's facility busy. Both Infante and Ceja are part of a burgeoning workforce developing the building blocks for neural networks; they previously worked at Scale, another AI data annotation firm, before joining Encord.

Ceja had worked at a waste management company where his interest in technology found him in charge of keeping a robotic trash sorter in good working order. Now, as the Jenga tower topples, he says he enjoys the challenge of solving training tasks for robots.

"It's something new every day!"

The Road Ahead

The brain wave experiment is still in its early stages. Encord and Zander Labs need to build a substantial dataset, run it through customer robotics models, and evaluate whether it actually improves performance before deciding whether to scale it up. The science is promising, but it's unproven.

What is clear, however, is that the data bottleneck is real. The companies that figure out how to generate high-quality physical training data efficiently will have a significant advantage in the race to build useful, general-purpose robots. The brain wave headset in San Leandro is just one experiment among many but it represents a direction that could fundamentally change how we teach robots to understand and interact with the physical world.

The circus of AI innovation is always packing up and moving away. But in a warehouse in San Leandro, the circus is just getting started one Jenga block, one brain wave, and one ethernet cable at a time.