Why Humanoid Robots Still Struggle Beyond Polished Demos

While generative AI rapidly advanced by training on the vast amount of text already available online, humanoid robots face a much different challenge: gaining enough real-world experience to operate reliably in everyday environments. Experts say the biggest obstacle for physical AI is not intelligence itself, but the difficulty of collecting the enormous amounts of sensorimotor data needed to train machines for real-world tasks.

Robot demonstration videos often showcase machines folding laundry, pouring drinks, or handling objects with apparent ease. However, these carefully edited clips rarely show failed attempts, such as missing a handle, damaging an object, or becoming confused by minor changes in the environment. Those failures highlight a fundamental issue in robotics: machines frequently struggle with situations that fall outside their training experience.

Large language models benefited from decades of internet content created by billions of people. Trillions of words were already available for companies to collect, process, and use for training. That existing data foundation allowed generative AI systems to scale quickly. Robotics, by contrast, lacks an equivalent resource. There is no vast, internet-scale repository of physical interactions that captures the countless ways humans manipulate objects, recover from mistakes, and adapt to changing conditions.

One of the largest efforts to address this challenge is Open X-Embodiment, a project that has gathered more than one million real robot trajectories spanning 22 robot embodiments and 527 skills. Although significant by robotics standards, the dataset remains small compared with the massive volumes of text used to train modern AI models. Every trajectory also requires a physical robot performing real actions in a real-world setting.

To expand training data, robotics companies increasingly rely on teleoperation systems. Human operators use technologies such as virtual reality equipment and exoskeletons to remotely control robots while performing repetitive tasks. These actions are recorded and later used to train robotic systems, creating the data needed for future autonomous operation.

Simulation has also become a critical tool. Companies including NVIDIA allow robots to practice tasks millions of times in virtual environments without the costs and risks associated with physical hardware. While simulation accelerates learning, experts note that virtual environments still struggle to accurately reproduce many real-world conditions, including deformable objects, cluttered spaces, inconsistent lighting, damaged packaging, and other unpredictable factors.

According to observers involved in robotics deployments, these so-called “long-tail” scenarios remain one of the industry’s biggest challenges. A robot may perform impressively during demonstrations yet fail when confronted with an unexpected but routine situation in a warehouse, factory, hospital, or kitchen.

As a result, experts recommend evaluating robotic systems based on their real-world experience rather than polished demonstrations. Key questions include how many hours of real-world task data were used during training, how the system handles edge cases, and how often human intervention is still required. Performance that appears autonomous may sometimes depend on teleoperation, scripted resets, or highly controlled environments.

Industry observers argue that many business leaders assess robots as they would software, focusing on whether the underlying AI model is becoming smarter. In practice, the more important measure may be whether the system has accumulated enough real-world experience to operate successfully in unpredictable environments where exceptions are common.

The outlook for humanoid robotics remains promising, but experts suggest progress will depend less on marketing announcements and more on the slow, costly process of gathering physical-world training data. Unlike language models, which benefited from decades of online content, robots must acquire experience through direct interaction with the world, one task, mistake, and recovery at a time.