Multimodality Driven Impedance-Based Sim2Real Transfer Learning for Robotic Multiple Peg-in-Hole Assembly

【Author】 Chen, Wenkai; Zeng, Chao; Liang, Hongzhuo; Sun, Fuchun; Zhang, Jianwei

【Source】IEEE TRANSACTIONS ON CYBERNETICS

【影响因子】19.118

【Abstract】Robotic rigid contact-rich manipulation in an unstructured dynamic environment requires an effective resolution for smart manufacturing. As the most common use case for the intelligence industry, a lot of studies based on reinforcement learning (RL) algorithms have been conducted to improve the performances of single peg-in-hole assembly. However, existing RL methods are difficult to apply to multiple peg-in-hole issues due to more complicated geometric and physical constraints. In addition, previously limited solutions for multiple peg-in-hole assembly are hard to transfer into real industrial scenarios flexibly. To effectively address these issues, this work designs a novel and more challenging multiple peg-in-hole assembly setup by using the advantage of the Industrial Metaverse. We propose a detailed solution scheme to solve this task. Specifically, multiple modalities, including vision, proprioception, and force/torque, are learned as compact representations to account for the complexity and uncertainties and improve the sample efficiency. Furthermore, RL is used in the simulation to train the policy, and the learned policy is transferred to the real world without extra exploration. Domain randomization and impedance control are embedded into the policy to narrow the gap between simulation and reality. Evaluation results demonstrate the effectiveness of the proposed solution, showcasing successful multiple peg-in-hole assembly and generalization across different object shapes in real-world scenarios.

【Keywords】Task analysis; Robots; Impedance; Visualization; Metaverse; Shape; Service robots; Impedance control; multimodal learning; RL; robotic multiple peg-in-hole assembly; sim2real-based transfer learning

【发表时间】2023 2023 SEP 15

【收录时间】2023-10-15

【文献类型】实验仿真

【主题类别】

区块链应用-虚拟经济-元宇宙