FOMI: A Force-Feedback Manipulation Interface for Teaching Contact-Rich Manipulation

Embodied AI and Robotics Imitation Learning CFCIC Tactile Sensing
2026年10月14日
无需机器人参与、仅依靠手持式交互设备即可规模化采集人类操作示范,从而用于策略学习。然而,现有手持式交互设备通常仅能提供有限的、不够直观的力反馈,致使人类操作者在采集富含接触操作的示范时面临较大挑战。为解决这一问题,我们提出了FOMI——一种基于剪刀式机械结构的力反馈操作接口。该接口通过驱动连杆直接将物体交互力传递至操作者双手,全程无需依赖力传感器。这种直观的力反馈机制,可支持操作者在无需机器人参与的示范采集过程中实时调整动作。围绕FOMI,我们构建了一套完整的示范采集系统,集成了传感器标定、数据采集与多源信号同步等功能,全面支撑无机器人介入的示范采集。此外,通过将FOMI的姿态变化及夹爪开合状态映射为机器人末端执行器的位姿与夹爪控制指令,该系统亦可支持遥操作功能。在四项富含接触操作的任务中,FOMI在无机器人参与的示范采集模式下,相较最优基线方法效率提升约29%;在遥操作模式下,效率提升约18%。利用FOMI在上述两种模式下所采集的示范数据训练得到的策略,均能在具有挑战性的富含接触操作的操控任务中实现高成功率。
Robot-free collection with handheld interfaces enables scalable acquisition of human demonstrations for policy learning. However, existing handheld interfaces often provide limited intuitive force feedback, making contact-rich demonstration collection challenging for human operators. To address this, we present FOMI, a force-feedback manipulation interface with a scissor-shaped mechanism, which transmits object interaction forces directly to the operator's hands through actuation links without force sensors. This intuitive force feedback supports real-time action adjustment during robot-free collection of contact-rich demonstrations. Around FOMI, we build a demonstration collection system that integrates sensor calibration, data acquisition, and synchronization for robot-free collection. By further mapping pose changes and gripper opening of FOMI to robot end-effector poses and gripper commands, the system also supports teleoperation. Across four contact-rich tasks, FOMI improves collection efficiency by approximately 29% and 18% over the best baselines in robot-free collection and teleoperation, respectively. Demonstrations collected with FOMI in both modes enable policies to perform challenging contact-rich manipulation tasks with high success rates.
许愿