HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
Shuanghao Bai, Meng Li, Xinyuan Lv, Jiawei Wang, Xinhua Wang, Fei Liao, Chengkai Hou, Langzhe Gu
Abstract
Humans achieve complex manipulation through coordinated whole-body control, whereas most Vision-Language-Action (VLA) models treat robot body parts largely independently, making high-DoF humanoid control challenging and often unstable. We present HEX, a state-centric framework for coordinated manipulation on full-sized bipedal humanoid robots. HEX introduces a humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments, and incorporates a Mixture-of-Experts Unified Proprioceptive Predictor to model whole-body coordination and temporal motion dynamics from large-scale multi-embodiment trajectory data. To efficiently capture temporal visual context, HEX uses lightweight history tokens to summarize past observations, avoiding repeated encoding of historical images during inference. It further employs a residual-gated fusion mechanism with a flow-matching action head to adaptively integrate visual-language cues with proprioceptive dynamics for action generation. Experiments on real-world humanoid manipulation tasks show that HEX achieves state-of-the-art performance in task success rate and generalization, particularly in fast-reaction and long-horizon scenarios.
AI Impact Assessments
(3 models)Scientific Impact Assessment: HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation
1. Core Contribution
HEX presents a Vision-Language-Action (VLA) framework specifically designed for coordinated whole-body manipulation on full-sized bipedal humanoid robots. The central insight is that effective humanoid control requires explicit modeling of inter-body-part coordination, rather than treating joints independently as most VLA models do. The paper introduces four key technical contributions:
2. Methodological Rigor
The experimental evaluation is thorough and well-structured, covering multiple dimensions:
Strengths in evaluation design:
Concerns about rigor:
3. Potential Impact
Direct impact on humanoid robotics: HEX addresses a genuine gap — most VLA models were designed for fixed-base manipulators and struggle with the coordination demands of bipedal humanoids. The body-part-aligned representation and predictive proprioceptive modeling are practical solutions that could become standard components in humanoid VLA architectures.
Cross-embodiment learning: The canonical body-part slot formulation with missing-part tokens is an elegant solution for heterogeneous embodiment learning. This could generalize beyond humanoids to other multi-limbed robots.
Broader VLA community: The history query cache mechanism is a generally applicable efficiency technique that avoids the quadratic cost of encoding multiple historical frames. The residual-gated fusion approach for multi-modal conditioning could influence action generation architectures more broadly.
Limitations on impact: The hierarchical design (high-level VLA + low-level RL controller) means HEX's whole-body coordination is partially dependent on pre-trained skill-specific controllers (standing, walking, kneeling), limiting true end-to-end generalization. The reliance on specific hardware platforms and proprietary data collection infrastructure reduces immediate reproducibility.
4. Timeliness & Relevance
This work is highly timely. The humanoid robotics field is experiencing rapid growth, with companies and labs racing to deploy full-sized humanoids. The gap between VLA models designed for tabletop manipulation and the demands of whole-body humanoid control is a recognized bottleneck. HEX directly addresses this by proposing principled architectural changes rather than simply scaling existing approaches.
The paper's "review-and-forecast" paradigm — combining visual history summarization with future state prediction — is a conceptually appealing framework that resonates with growing interest in world models and predictive control in robotics.
5. Strengths & Limitations
Key Strengths:
Notable Weaknesses:
Additional Observations
The paper's dataset contribution (12M+ frames across 7 embodiments) is valuable but not fully open-sourced. The routing pattern analysis showing phase-dependent expert selection is one of the more interesting empirical findings, suggesting that MoE architectures can learn meaningful temporal structure in manipulation sequences. The failure analysis via Sankey diagrams is a useful visualization that the community could adopt.
Generated Apr 10, 2026
Comparison History (36)
HEX addresses the broader and more impactful problem of cross-embodiment whole-body manipulation for humanoid robots using VLA models, combining vision-language-action integration with scalable multi-embodiment learning. Its contributions span multiple high-impact areas: cross-embodiment transfer, mixture-of-experts architectures, and real-world humanoid manipulation. ParkourFormer, while technically strong with impressive parkour results, addresses a narrower locomotion problem. HEX's framework for coordinated manipulation with language grounding has wider applicability and aligns with the rapidly growing VLA/foundation model trend, giving it greater potential for broad scientific influence.
Paper 2 has higher potential impact due to its release of the largest embodied CoT corpus to date (978K trajectories), which will likely become a foundational resource for the embodied AI community. Furthermore, its novel approach of using CoT as representation-shaping supervision rather than a mandatory test-time prefix elegantly solves the compounding error and latency bottlenecks in current VLA models. While Paper 1 offers excellent innovations for humanoid control, Paper 2's massive open-source dataset and architectural insights provide broader, field-wide utility across general robotic manipulation.
Paper 2 addresses a fundamental and notoriously difficult bottleneck in robotic manipulation: the integration of high-frequency tactile feedback. By introducing a large-scale 100-hour tactile dataset and a novel temporal VQ-VAE encoder with a variable-rate architecture, it provides resources and methods that can broadly benefit the entire manipulation community. While Paper 1 presents a strong framework for humanoid control, Paper 2's foundational dataset and resolution of tactile-integration challenges offer a higher potential for widespread, cross-disciplinary impact in achieving human-level dexterity.
Paper 1 tackles a critical bottleneck in robotics—whole-body coordination for high-DoF real-world humanoids—using a novel cross-embodiment VLA framework. Its deployment on real bipedal robots offers significant, immediate impact in a highly active field. While Paper 2 presents an interesting conceptual paradigm for continuous learning, its reliance on simulated tasks makes its immediate real-world impact less tangible compared to Paper 1's concrete methodological advancements.
Paper 1 addresses the highly challenging and cutting-edge problem of whole-body manipulation for bipedal humanoids using Vision-Language-Action models. Its cross-embodiment approach and focus on foundation models offer broader applicability and alignment with current high-impact trends in AI and robotics. Paper 2, while methodologically rigorous, focuses on a narrower and more traditional problem (Learning from Demonstration on a 7-DOF arm), limiting its comparative breadth of impact and novelty.
Paper 2 likely has higher scientific impact due to greater methodological novelty (universal humanoid-aligned state, MoE proprioceptive dynamics, efficient temporal visual context, and fusion/action-head design) and broader applicability across embodiments and whole-body humanoid manipulation—an increasingly central and timely area in robotics/physical AI. It reports state-of-the-art real-world results and generalization, suggesting strong practical relevance and cross-field influence (representation learning, control, multimodal ML). Paper 1 is valuable and rigorous as a systems measurement study, but is more incremental and primarily impacts robotics systems/offloading design rather than core capability advances.
Paper 1 addresses a fundamental bottleneck in the rapidly growing field of humanoid robotics: coordinated whole-body control and cross-embodiment learning. By introducing a universal state representation and an MoE predictor, it offers a scalable architectural advancement for high-DoF systems. While Paper 2 presents a highly practical solution for viewpoint robustness, Paper 1 represents a deeper architectural innovation that pushes the boundaries of what Vision-Language-Action models can achieve in complex, heterogeneous physical embodiments, likely spurring more follow-up research in foundational robot learning.
HEX addresses a fundamental challenge in humanoid robotics—whole-body coordinated manipulation—with a novel framework combining mixture-of-experts proprioceptive prediction, cross-embodiment learning, and efficient temporal reasoning. Its contributions span multiple high-impact areas (VLA models, humanoid control, cross-embodiment transfer) with demonstrated state-of-the-art results. While AgiPIX provides a valuable open-source drone platform for indoor inspection, it is more incremental as a systems integration contribution. HEX's novelty in coordinated whole-body humanoid control and scalable cross-embodiment learning has broader impact potential across robotics and AI.
Paper 2 (HEX) likely has higher scientific impact due to a broadly relevant, technically substantial framework for whole-body humanoid manipulation—an urgent, high-impact challenge in robotics and VLA. Its universal humanoid-aligned state, multi-embodiment learning, MoE proprioceptive dynamics, and efficient temporal context mechanisms are broadly reusable across platforms and tasks, with strong real-world applicability and cross-field relevance (robot learning, control, representation learning). Paper 1 is novel in calibrated LLM communication for multi-agent EQA and adds a benchmark, but its scope is narrower and more simulation/benchmark-centric.
Paper 2 presents a novel framework addressing a critical bottleneck in robotics: high-DoF humanoid whole-body control and cross-embodiment learning. Its technical innovations, including humanoid-aligned universal state representation and a mixture-of-experts predictor, offer significant advancements over existing models. While Paper 1 is a valuable survey, Paper 2's direct methodological contributions to scalable, real-world robotic manipulation and its alignment with the rapidly growing field of generalist humanoid robots give it a higher potential for disruptive scientific impact.