摘要
随着深度学习的发展,基于卷积神经网络(CNN)的目标检测方法取得巨大成功。现有的基于CNN的目标检测模型通常采用单一模态的RGB图像进行训练和测试,但在低光照环境下,检测性能显著下降。为解决此问题,提出了一种基于YOLOv5构建的多模态目标检测网络模型,将RGB图像和热红外图像相结合,以充分利用多模态特征融合信息,从而提升目标检测精度。为了实现多模态特征信息的有效融合,提出了一种多模态自适应特征融合(MAFF)模块。该模块通过自适应地选择不同模态特征并利用各模态间的互补信息,实现多模态特征融合。实验结果表明:所提算法能有效融合不同模态的特征信息,从而显著提高检测精度。
With the advancement of deep learning,object detection methods based on convolutional neural networks(CNNs)have achieved tremendous success.Existing CNN-based object detection models typically employ single-modal RGB images for training and testing;however,their detection performance is significantly degraded in low-light conditions.To address this issue,a multimodal object detection network model built on YOLOv5 is proposed,which integrates RGB and thermal infrared imagery to fully exploit the information provided by the fusion of multi-modal features,increasing the object detection accuracy.To achieve effective fusion of multimodal feature information,a multimodal adaptive feature fusion(MAFF)module is introduced.It facilitated multimodal feature fusion by adaptively selecting diverse modal features and exploiting the complementary information between modalities.The experimental results indicate the efficacy of the proposed algorithm for seamlessly merging features from distinct modalities,which significantly increases the detection accuracy.
作者
高小强
常侃
凌铭阳
银梦雨
Gao Xiaoqiang;Chang Kan;Ling Mingyang;Yin Mengyu(School of Computer and Electronic Information,Guangxi University,Nanning 530004,Guangxi,China;Guangxi Key Laboratory of Multimedia Communications and Network Technology,Nanning 530004,Guangxi,China)
出处
《激光与光电子学进展》
CSCD
北大核心
2023年第24期100-109,共10页
Laser & Optoelectronics Progress
基金
国家自然科学基金(62171145)。
作者简介
通信作者:常侃,pandack0619@163.com。
