Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation (CoP)
Researchers introduce Center-of-Pressure (CoP), a tactile representation grounded in physical principles designed to bridge the sim-to-real gap in contact-rich dexterous manipulation. CoP preserves dense contact information while remaining robust for sim-to-real transfer, supported by a differentiable-dynamics-based sensor calibration scheme that estimates taxel orientations without ground-truth force measurements. Evaluated on peg-in-hole insertion and ball balancing tasks, CoP-conditioned policies achieve zero-shot sim-to-real transfer on a multi-fingered robotic hand, outperforming binary-contact and raw-taxel baselines. An emergent finding is that CoP-conditioned policies implicitly encode task-relevant physical properties such as object mass.
Related guides (2)
Related events (8)
Mana framework achieves zero-shot sim-to-real transfer for dexterous articulated tool manipulation
Researchers introduce Mana (Manipulation Animator), a sim-to-real framework that reframes dexterous robotic manipulation as an animation problem using a coarse-to-fine pipeline of procedurally-generated grasp keyframes, motion planning, and reinforcement learning. The system requires minimal human input (under one minute per tool) and achieves zero-shot sim-to-real transfer across four articulated tools with varying joint types and scales. The work addresses a longstanding gap in dexterous robotics where articulated tool use—requiring coordination of internal degrees of freedom and contact-rich interactions—has been underexplored relative to rigid object manipulation.
DexHoldem: A Real-World Benchmark for Dexterous Embodied Agents Using Texas Hold'em Manipulation
DexHoldem is a new system-level benchmark for evaluating dexterous embodied agents on a ShadowHand robot performing Texas Hold'em card manipulation tasks. It provides 1,470 teleoperated demonstrations across 14 manipulation primitives, a physical policy benchmark, and an agentic perception benchmark for structured game-state recovery. Top performers include π₀.₅ at 61.2% task completion and Claude Opus 4.7 at 34.3% strict perception accuracy, with GPT 5.5 achieving 66.8% field-wise accuracy. The benchmark exposes gaps between isolated visual sub-capabilities and full closed-loop embodied decision-making.
FACTR 2: Neural External Torque Estimation enables force-aware robot policy learning without dedicated sensors
Researchers introduce NEXT (Neural External Torque Estimation), a data-driven method that estimates external joint torques on commodity robot arms without dedicated force sensors, training in one minute from ten minutes of free-motion data. Combined with Force-Informed Re-Sampling Training (FIRST), which up-samples contact segments during behavior cloning, the system outperforms prior force-aware policies by over 17% in task progress across five long-horizon manipulation tasks. The work lowers the hardware barrier for contact-rich robot manipulation by bringing force-feedback teleoperation and policy learning to off-the-shelf arms.
DynaFLIP: Dynamics-Aware Multimodal Pre-Training for Robot Manipulation Perception
DynaFLIP is a pre-training framework that injects motion understanding into visual encoders for robot manipulation by constructing image-language-3D flow triplets from human and robot videos. The method encourages tri-modal alignment via simplex-volume minimization in a shared hyperspherical space, combined with cosine regularization and contrastive objectives. The resulting dynamics-aware visual backbone consistently outperforms baselines across diverse downstream policies including VLAs, with gains up to +22.5% in out-of-distribution scenarios. The work argues that robot generalization requires encoding how the world changes under action, not just static scene content.
HITL-D: Human-In-The-Loop Diffusion for Shared Control in Robotic Manipulation
HITL-D is a shared control framework that combines diffusion-based policies with human teleoperation for robotic manipulation tasks. The system autonomously updates end-effector orientation conditioned on scene point clouds and Cartesian position, reducing the number of joystick axes operators must manage. A 12-participant user study found 40% faster task completion, 37% lower perceived workload, and improved subjective ratings versus traditional teleoperation. The work addresses a relatively unexplored intersection of diffusion policy methods and human-in-the-loop control.
Geometric Action Model (GAM) repurposes geometric foundation models for 3D-aware robot manipulation
Researchers propose the Geometric Action Model (GAM), a language-conditioned robot manipulation policy that splits a pretrained geometric foundation model (GFM) to serve simultaneously as an observation encoder, causal future predictor, and action decoder. Unlike existing vision-language-action models that operate on 2D image frames, GAM explicitly incorporates 3D geometric priors for contact-rich manipulation. The approach claims improvements in accuracy, robustness, speed, and model size over foundation-model-scale baselines across simulation and real-robot benchmarks.
Learning Dexterity: OpenAI Trains Robot Hand for Physical Object Manipulation
OpenAI announced the training of a human-like robot hand capable of manipulating physical objects with what they describe as unprecedented dexterity. The system uses reinforcement learning to develop fine motor control in a dexterous robotic hand. This work represents an early milestone in OpenAI's robotics research program, predating their later Dactyl work on solving Rubik's cubes.
Sim-to-real transfer of robotic control with dynamics randomization
OpenAI published research on transferring robotic control policies trained in simulation to real-world robots using dynamics randomization. The technique involves varying physical parameters during simulation training so that the real world appears as just another variation, enabling zero-shot sim-to-real transfer. This was an early foundational contribution to the sim-to-real robotics research thread.