摘要
识别非驾驶行为是提高驾驶安全性的重要手段之一。目前基于骨架序列和图像的融合识别方法具有计算量大和特征融合困难的问题。针对上述问题,本文提出一种基于多尺度骨架图和局部视觉上下文融合的驾驶员行为识别模型(skeleton-image based behavior recognition network,SIBBR-Net)。SIBBR-Net通过基于多尺度图的图卷积网络和基于局部视觉及注意力机制的卷积神经网络,充分提取运动和外观特征,较好地平衡了模型表征能力和计算量间的关系。基于手部运动的特征双向引导学习策略、自适应特征融合模块和静态特征空间上的辅助损失,使运动和外观特征间互相引导更新并实现自适应融合。最终在Drive&Act数据集进行算法测试,SIBBR-Net在动态标签和静态标签条件下的平均正确率分别为61.78%和80.42%,每秒浮点运算次数为25.92G,较最优方法降低了76.96%。
Non-driving behavior identification is one of the important ways to improve the safety of driving.The current recognition method based on skeleton sequence and image fusion has the problems of large model calcu⁃lation and the difficulty of feature fusion.To address the above problems,the skeleton-image based behavior recogni⁃tion network(SIBBR-Net)is proposed in this paper,which is based on the multi-scale skeleton graph and the local visual context.SIBBR-Net fully extracts motion and appearance features through a graph convolution network based on multi-scale skeleton graphs and a convolutional neural network based on local vision and attention mechanisms,and better balances the relationship between model representation capabilities and model calculation.The feature bi⁃directional guided learning strategy based on hand motion,an adaptive feature fusion module and an auxiliary loss on the static feature space can guide mutual guidance and updating between motion and appearance features to achieve adaptive fusion.SIBBR-Net is finally tested on the Drive&Act dataset,and the average accuracy is 61.78%for dynamic labels and 80.42%for static labels.The Floating-point Operations per Second(FLOPS)of SIB⁃BR-Net is 25.92G,which is 76.96%lower than that of the optimal method.
作者
胡宏宇
黎烨宸
张争光
曲优
何磊
高镇海
Hu Hongyu;Li Yechen;Zhang Zhengguang;Qu You;He Lei;Gao Zhenhai(Jilin University,State Key Laboratory of Automotive Simulation and Control,Changchun 130022)
出处
《汽车工程》
EI
CSCD
北大核心
2024年第1期1-8,28,共9页
Automotive Engineering
基金
吉林省自然科学基金(20210101064JC)
国家自然科学基金(52272417)
新能源智能汽车关键技术研发及产业化项目(TC210H02S)
大学生创新创业训练计划项目(X202310183158)资助。
作者简介
通信作者:何磊,教授,博士,E-mail:jlu_helei@jlu.edu.cn。
