摘要
针对柔性物体操纵问题,提出了一种视/触觉融合的柔性物体抓取力估计方法——多感知局部增强Transformer(MSLET)。该方法利用模型学习每种传感器模态的低维特征,推断待抓取物体的物理特征,融合各模态的物理特征向量,用于预测抓取结果,并结合安全抓取物体的经验,推断最佳抓取力。首先,提出了用于提取视觉图像和触觉图像浅层特征的Feature-to-Patch模块,它利用2种图像的浅层特征提取图像块,进而得到它们的边缘特征,充分学习数据的特征信息,更好地推断物体的物理特征。其次,提出了用于增强局部特征的Local-Enhanced模块,对多头自注意力机制生成的图像块进行深度可分离卷积处理,以此增强局部性特征处理,促进空间维度上相邻标记之间的相关性,提高抓取结果的预测准确率。最后,对比实验表明,本文算法在保证运行效率的同时,将抓取准确率相较于当前最优模型提高了10.19%,证明该算法能够有效估计柔性物体抓取力。
To address the manipulation problem of flexible objects,a visual-tactile fusion method for estimating the grasping force on flexible objects is proposed,named the MultiSense Local-Enhanced Transformer(MSLET).This approach uses a model to learn low-dimensional features from each sensor modality,infers the physical characteristics of the grasped object,and integrates these modality-specific feature vectors to predict the grasping result.By leveraging knowledge of safe grasping practices,the optimal grasping force is inferred.Firstly,the Feature-to-Patch module is developed to extract shallow features from both visual and tactile images.This module generates image patches from these shallow features,capturing their edge characteristics,thus effectively learning the feature information from data and inferring the physical properties of the grasped objects.Secondly,the Local-Enhanced module is proposed to enhance local features.Depth-wise separable convolution is applied to the image patches produced by the multi-head self-attention mechanism,to enhance the local feature processing.This increases the correlation between adjacent tokens in the spatial dimension,improving the prediction accuracy of grasping results.Finally,comparative experiments demonstrate that the proposed algorithm improves the grasping accuracy by10.19%over the state-of-the-art models while ensuring operational efficiency,thereby proving its effectiveness in estimating the grasping force on flexible objects.
作者
吴培良
李瑶
牛明月
陈雯柏
高国伟
WU Peiliang;LI Yao;NIU Mingyue;CHEN Wenbai;GAO Guowei(School of Information Science and Engineering,Yanshan University,Qinhuangdao 066004,China;The Key Laboratory for Computer Virtual Technology and System Integration of Hebei Province,Qinhuangdao 066004,China;School of Automation,Beijing Information Science and Technology University,Beijing 100192,China)
出处
《机器人》
EI
CSCD
北大核心
2024年第5期544-553,共10页
Robot
基金
国家自然科学基金(U20A20167,62276028)
河北省自然科学基金(F2021203079)
河北省中央引导地方科技发展资金(236Z1811G)
河北省创新能力提升计划(22567626H)。
关键词
视/触觉融合
抓取力估计
物理特征嵌入
visual-tactile fusion
grasping force estimation
physical feature embedding
作者简介
吴培良(1981-),男,博士,教授。研究领域:家庭服务机器人工具认知,机器人抓取,强化学习;通信作者:李瑶(1994-),女,博士生。研究领域:视/触觉融合的机器人操作,深度学习,ysuly@stumail.ysu.edu.cn。
