摘要
在复杂人机协作环境中,视觉遮挡和数据丢失问题使单得靠Kinect深度相机难以准确预测人体动作。为解决这一问题,提出了一种基于多模态信息融合与改进长短期记忆(LSTM)网络——BD-LSTM网络的三维人体动作预测方法。该方法融合了Kinect三维骨架数据与OpenPose提取的RGB图像关节点信息,并引入骨骼长度不变性和方向一致性约束以优化动作序列的映射关系。在四种典型人机协作情境下对本文模型进行了实验验证,比较了典型的预测模型LSTM、双向LSTM(BiLSTM)、循环神经网络(RNN)和本文BD-LSTM模型的预测误差,实验结果表明,BD-LSTM的预测误差最小,进一步证明了本文模型在预测精度和稳定性方面的优越性。
In complex human-robot collaboration environments,visual occlusion and data loss make it difficult to accurately predict human motion using only the Kinect depth camera.To solve this problem,a 3D human motion prediction method based on multimodal information fusion and improved long short-term memory(LSTM)network—BD-LSTM network is proposed.This method fuses Kinect 3D skeleton data with RGB image joint point information extracted by OpenPose and introduces bone length invariance and direction consistency constraints to optimize the mapping relationship of action sequences.The proposed model is experimentally verified in four typical human-robot collaborative scenarios,and the prediction errors of typical prediction models LSTM,bidirectional LSTM(BiLSTM),recurrent neural network(RNN)and the proposed BD-LSTM model are compared.The experimental results show that the prediction error of BD-LSTM is the smallest,which further proves the superiority of the proposed model in terms of prediction accuracy and stability.
作者
彭巍
唐友康
覃璐
董元发
周彬
安友军
PENG Wei;TANG Youkang;QIN Lu;DONG Yuanfa;ZHOU Bin;AN Youjun(College of Mechanical and Power Engineering,China Three Gorges University,Yichang 443002,China)
出处
《传感器与微系统》
2025年第9期138-143,共6页
Transducer and Microsystem Technologies
基金
国家自然科学基金资助项目(52075292)
湖北省自然科学基金资助项目(2023AFB1116)
湖北省自然科学基金资助项目(2022CFB798)。
关键词
人机协作
多模态信息融合
骨骼优化
动作预测
human-robot collaboration
multimodal information fusion
skeleton optimization
motion prediction
作者简介
彭巍(1986-),男,博士,副教授,主要研究领域为人机协作、智能制造;通讯作者:董元发(1988-),男,博士,教授,博士研究生导师,主要研究领域为智能装备与系统、数字孪生与智能制造、人机共融。
