ForceBand: Learning Forceful Manipulation with sEMG

Embodied AI and Robotics Imitation Learning CFCIC
2026年06月24日
人类示范是学习机器人操作策略的一种可扩展数据来源。然而,人类示范数据的常见来源(例如动作捕捉轨迹和网络视频)主要记录运动轨迹与外观信息,却缺失了对力敏感型操作至关重要的接触力信息。本文提出ForceBand——一种低成本、佩戴于手腕的表面肌电信号(sEMG)系统,可将人体肌肉活动转化为富含力信息的示范数据。我们首先采集了一个时长10小时的多模态数据集,涵盖第一人称视角视频、sEMG信号、惯性测量单元(IMU)数据以及指尖接触力测量结果,覆盖多种操作任务与不同物体。基于该数据集,我们预训练了一个EMG2Force模型,该模型能够根据sEMG与IMU信号预测每一根手指所受的力。在完成简短的用户个性化校准后,用户仅需佩戴ForceBand并录制视频,即可采集目标任务的示范数据;随后,EMG2Force模型会为这些示范自动标注每根手指的力变化轨迹,从而生成具备力信息增强的高质量示范数据,用于机器人策略学习。实验结果表明,ForceBand在恢复精细指尖交互方面表现优异,其力预测误差比基于视觉的基线方法降低超过50%;在涉及物体特异性力控的“抓取—捏握—放置”任务中,针对形状、尺寸与重量各异的物体,ForceBand实现了87%的成功率。项目网站:https://forceband-emg.github.io
Human demonstrations are a scalable data source for learning robot manipulation policies. However, common sources of human demonstration data, such as motion-capture trajectories and internet videos, capture mostly motion and appearance while missing the contact forces that are critical for force-sensitive manipulation. In this paper, we introduce ForceBand, a low-cost wrist-worn sEMG system that turns human muscle activity into force-enriched demonstrations. We first collect a 10-hour multimodal dataset containing egocentric video, sEMG, IMU, and fingertip force measurements across diverse actions and objects. Using this dataset, we pre-train an EMG2Force model that predicts per-finger forces from sEMG and IMU signals. After a short user-specific calibration, users can collect target-task demonstrations using only ForceBand and video; EMG2Force then labels these demonstrations with per-finger force traces, producing force-augmented demonstrations for robot policy learning. Experiments show that ForceBand recovers fine-grained fingertip interactions with over 50% lower force prediction error than vision-based baselines and achieves an 87% success rate on pick, squeeze, and place tasks that require object-specific force control across objects with diverse shapes, sizes, and weights. Project website: https://forceband-emg.github.io
许愿