摘要
针对单模态眼底图像提取眼底特征的局限性,提出一个基于多层次多尺度注意力融合网络的多模态眼底疾病诊断模型.首先,分别针对彩色眼底图像和视网膜光学相干断层成像设计多层次注意力网络和多尺度注意力网络,并在特征层进行融合得到融合特征;其次,将两种模态的损失函数加权,与融合特征的损失函数相加,提取模态的独特和互补信息,以提高眼底疾病诊断的准确率.在数据集MMC-AMD和GAMMA上进行评估的实验结果表明,该模型优于当前主流模型,诊断效果优越.
Aiming at the limitations of extracting retinal features from single-mode retinal images,we proposed a multi-modal retinal disease diagnosis model based on multi-level and multi-scale attention fusion network.Firstly,the multi-level attention network and multi-scale attention network were designed for color retinal images and retinal optical coherence tomography respectively,and the fusion features were obtained by merging at the feature layer.Secondly,the weighted loss function of the two modes and the loss function of the fusion features were added to extract the unique and complementary information of the two modes in order to improve the accuracy of retinal disease diagnosis.The results of evaluation experiments on the MMC-AMD dataset and GAMMA dataset show that the proposed model outperforms the current mainstream models and has superior diagnostic effect.
作者
郭晓新
杨梅
杨广奇
董洪良
徐海啸
GUO Xiaoxin;YANG Mei;YANG Guangqi;DONG Hongliang;XU Haixiao(College of Computer Science and Technology,Jilin University,Changchun 130012,China;Key Laboratory of Symbolic Computation and Knowledge Engineerin g of Ministry of Education,Jilin University,Changchun 130012,China)
出处
《吉林大学学报(理学版)》
北大核心
2025年第3期783-794,共12页
Journal of Jilin University:Science Edition
基金
国家自然科学基金(批准号:82071995)
吉林省科技发展计划项目(批准号:20220201141GX)。
关键词
医学图像分类
眼底疾病诊断模型
多模态分类
注意力机制
medical image classification
retinal disease diagnosis model
multi-modal classification
attention mechanism
作者简介
第一作者:郭晓新(1974-),男,汉族,博士,教授,从事机器视觉和医疗影像学的研究,E-mail:guoxx@jlu.edu.cn.
