Direction Matters: Learning Force Direction Enables Sim-to-Real Contact-Rich Manipulation
Yifei Yang, Anzhe Chen, Zhenjie Zhu, Kechun Xu, Yunxuan Mao, Yufei Wei, Lu Chen, Rong Xiong
Abstract
Sim-to-real transfer for contact-rich manipulation remains challenging due to the inherent discrepancy in contact dynamics. While existing methods often rely on costly real-world data or utilize blind compliance through fixed controllers, we propose a framework that leverages expert-designed controller logic for transfer. Inspired by the success of privileged supervision in kinematic tasks, we employ a human-designed finite state machine based position/force controller in simulation to provide privileged guidance. The resulting policy is trained to predict the end-effector pose, contact state, and crucially the desired contact force direction. Unlike force magnitudes, which are highly sensitive to simulation inaccuracies, force directions encode high-level task geometry and remain robust across the sim-to-real gap. At deployment, these predictions configure a force-aware admittance controller. By combining the policy's directional intent with a constant, low-cost manually tuned force magnitude, the system generates adaptive, task-aligned compliance. This tuning is lightweight, typically requiring only a single scalar per contact state. We provide theoretical analysis for stability and robustness to disturbances. Experiments on four real-world tasks, i.e., microwave opening, peg-in-hole, whiteboard wiping, and door opening, demonstrate that our approach significantly outperforms strong baselines in both success rate and robustness. Videos are available at: https://yifei-y.github.io/project-pages/DirectionMatters/.
AI Impact Assessments
(3 models)Scientific Impact Assessment: "Direction Matters: Learning Force Direction Enables Sim-to-Real Contact-Rich Manipulation"
1. Core Contribution
The paper identifies a key insight for sim-to-real transfer in contact-rich manipulation: while force magnitudes are highly sensitive to simulation inaccuracies, force directions are primarily determined by task geometry (contact manifold normals and tangents) and thus remain invariant across the sim-to-real gap. This observation motivates a framework where a policy trained purely in simulation predicts end-effector poses, contact states, and desired contact force directions. At deployment, these predictions configure a force-aware admittance controller, with force magnitudes specified as simple, manually-tuned scalars (one per contact phase).
The approach elegantly decomposes the force prediction problem: the geometry-dependent component (direction) is learned from simulation, while the dynamics-dependent component (magnitude) is specified through lightweight manual tuning. This resolves a fundamental tension in the field—contact-rich tasks need force reasoning, but simulated forces are unreliable.
2. Methodological Rigor
Theoretical foundation: The invariance analysis in Section III-C provides a clean geometric argument for why force directions transfer. The decomposition into tangent and normal force subspaces on the contact manifold is physically well-motivated. The stability analysis (Propositions 1-3) covers disturbance-free convergence, contact loss recovery, and ISS under perturbations, providing formal safety guarantees.
Experimental design: The evaluation across four diverse tasks (microwave opening, peg-in-hole, whiteboard wiping, door opening) covers different contact modalities—resistance overcoming, contact maintenance, constraint switching. The 25-trial protocol with randomized initial conditions and the separate 5-trial disturbance protocol are reasonable, though the sample sizes are modest. The disturbance evaluation protocol is thoughtfully designed to decouple robustness from initial contact success.
Weaknesses in rigor: The stability analysis assumes a linear spring environment model, which is a simplification. The theoretical invariance argument, while intuitive, assumes the contact manifold is well-defined and that task geometry is accurately captured in simulation—an assumption that could break down for deformable objects or complex contact geometries. The ablation study is conducted on only one task (whiteboard wiping), limiting generalizability of design choice conclusions.
3. Potential Impact
Practical significance: The framework is genuinely practical. Requiring only simulation data plus one manually-tuned scalar per contact phase is a significant reduction in real-world effort compared to methods requiring real demonstrations or real-world RL. The 91% overall success rate versus 67% for the best baseline is a substantial improvement.
Broader applicability: The direction-magnitude decomposition could become a design principle for other sim-to-real transfer problems involving physical interaction. The insight that geometric features of interaction are more transferable than dynamic ones has implications beyond manipulation—potentially extending to locomotion on varied terrains or tool use.
Limitations in scope: The framework currently handles only translational forces (acknowledged by authors). Tasks requiring precise torque control (knob turning, valve operation) are not addressed. The reliance on finite state machines for expert demonstration generation limits applicability to tasks with clearly defined contact phases. Continuous or multi-modal contact scenarios may be harder to handle.
4. Timeliness & Relevance
This work addresses a pressing bottleneck in robot learning. As foundation models for robotics (π₀, VLAs) advance rapidly, extending their capabilities from kinematic pick-and-place to contact-rich tasks is a critical frontier. The paper directly tackles why these large-scale sim-trained models fail at contact-rich manipulation and provides a principled solution. The choice to build on E2VLA and compare against π₀ positions this work at the intersection of foundation model deployment and classical control theory—a timely combination.
The growing interest in real-world deployment of learned policies makes the safety and robustness guarantees particularly relevant. The ISS stability result provides formal assurance that is increasingly demanded for industrial applications.
5. Strengths & Limitations
Key strengths:
Notable weaknesses:
Reproducibility: The paper provides extensive implementation details in appendices, uses publicly available simulation frameworks (IsaacLab), and references a project page with videos. The relatively simple controller design (standard admittance control with modifications) enhances reproducibility.
Overall Assessment
This is a well-executed paper that identifies a genuinely useful insight about the transferability of force direction versus magnitude. The combination of geometric analysis, practical system design, and formal stability guarantees is compelling. While the scope is limited to tasks with well-defined contact manifolds and translational forces, within this scope the results are strong and the approach is practical. The work makes a clear contribution to the sim-to-real transfer literature and provides a framework that could see adoption in industrial robotics applications.
Generated Apr 19, 2026
Comparison History (43)
Paper 1 introduces a notably novel and practical sim-to-real principle for contact-rich manipulation: predicting force direction (robust to sim inaccuracies) while using a lightweight, per-state scalar force magnitude with an admittance controller, plus stability/robustness analysis and multiple real-world task validations. This targets a core bottleneck in robotics (contact dynamics transfer) with broad applicability across manipulation tasks and control/learning stacks. Paper 2 is timely and strong, but its Transformer + predictive supervision for locomotion is a more incremental extension of existing sequence-modeling trends and is narrower in cross-domain impact than a general sim-to-real contact manipulation method.
Paper 2 demonstrates higher potential scientific impact due to its massive-scale contribution to the rapidly growing field of Vision-Language-Action (VLA) models. By introducing the largest embodied CoT corpus to date and a novel training paradigm that uses CoT as representation-shaping supervision rather than test-time generation, it addresses critical scalability and inference bottlenecks. While Paper 1 offers an elegant sim-to-real control approach, Paper 2's dataset release and foundational insights into LLM-based robotic reasoning will likely spur significantly broader derivative research across the AI and robotics communities.
Paper 1 addresses a critical and timely challenge in sim-to-real transfer for contact-rich manipulation with a novel insight—predicting force direction rather than magnitude to bridge the sim-to-real gap. It provides theoretical stability analysis and demonstrates strong empirical results across four diverse real-world tasks. The practical impact is high: it reduces real-world data requirements and enables robust contact-rich manipulation. Paper 2 presents a formal co-design framework for multi-robot systems, which is valuable but more incremental in its theoretical contribution and lacks real-world experimental validation, relying on case studies. Paper 1's actionable insight and demonstrated real-world applicability give it broader near-term impact.
Paper 1 addresses the critical and timely challenge of sim-to-real transfer for contact-rich manipulation with a novel insight—predicting force direction rather than magnitude to bridge the sim-to-real gap. This is highly practical, demonstrated on four real-world tasks, and offers theoretical stability guarantees. The approach is immediately applicable to robotics manipulation. Paper 2 presents a formal co-design framework for multi-robot systems, which is valuable but more theoretical/architectural in nature. Paper 1's concrete experimental validation, practical applicability, and novel conceptual contribution (force direction as a robust transferable signal) give it higher near-term scientific impact.
Paper 1 presents a more broadly applicable framework for sim-to-real transfer in contact-rich manipulation, addressing a fundamental challenge in robotics with a novel insight (force direction vs. magnitude for sim-to-real robustness). It demonstrates strong results across four diverse real-world tasks with theoretical stability analysis. Paper 2 makes a meaningful contribution to autonomous endovascular intervention using world models, but its impact is narrower (specific medical domain), results are incremental (moderate success rates, mixed in-vitro results), and the approach applies an existing method (TD-MPC2) rather than introducing a fundamentally new concept.
Paper 1 introduces a foundational shift in autonomous driving safety metrics, replacing the standard 1D time-to-collision with a 2D physically interpretable paradigm. Its extensive validation on real crash data and massive improvements over baselines suggest high potential to redefine industry safety benchmarks. Paper 2 offers a practical but narrower engineering solution for robotic sim-to-real transfer, relying on manual tuning and expert-designed state machines, which limits its fundamental scientific impact compared to the hyperparameter-free metric in Paper 1.
Paper 2 offers a more novel technical contribution: a sim-to-real framework that predicts contact force direction (robust across sim discrepancies) and uses it to parameterize a force-aware admittance controller, supported by theoretical stability/robustness analysis and multiple real-robot task demonstrations. This is timely and broadly useful for contact-rich manipulation, with clear real-world applicability and stronger methodological rigor than a codebase paper. Paper 1 (StarVLA) is valuable infrastructure for reproducibility and benchmarking, but its core contribution is integration/modularization rather than a new algorithmic insight, so its scientific impact is likely more incremental.
Paper 1 addresses the critical sim-to-real transfer problem for contact-rich manipulation with a novel insight—predicting force direction rather than magnitude—backed by theoretical stability analysis and demonstrated across four diverse real-world tasks. The approach is practical (requiring minimal real-world tuning), grounded in control theory, and tackles a fundamental bottleneck in robotic manipulation. Paper 2 presents a useful closed-loop policy framework with referring points, but its contribution is more incremental within the diffusion policy paradigm. Paper 1's insight about force direction robustness across sim-to-real gaps is more broadly impactful and transferable.
Paper 2 is likely to have higher scientific impact due to strong real-world applicability and demonstrated sim-to-real gains on multiple contact-rich manipulation tasks, a central robotics bottleneck. Its key insight—predicting force direction (robust to sim inaccuracies) while using simple tuned magnitudes—offers a practical, broadly adoptable recipe and integrates cleanly with existing controllers, enhancing transferability across labs and platforms. It also includes stability/robustness analysis and extensive real-robot validation. Paper 1 is technically novel and timely for generative control, but its impact may be narrower and depends on broader adoption of SB/diffusion policy stacks.
Paper 1 addresses the fundamental sim-to-real transfer challenge for contact-rich manipulation with a principled insight—force directions transfer better than magnitudes across the sim-to-real gap. It provides theoretical stability analysis, demonstrates across four diverse real-world tasks, and offers a practical framework combining privileged learning with admittance control. Paper 2 presents a creative approach to OOD robustness via world model imagination, but its impact is more incremental (improving existing diffusion policies). Paper 1's contribution is more broadly applicable to the robotics manipulation community and offers deeper methodological insight into contact dynamics transfer.