摘要
本文从空间维度和时间维度分析了场景弱变化区域和类型变化区域的融合规律、物理模型的差异性和效果上的互补性,提出了共享超分辨率的双分支(Shared Super-Resolution Dual-Branch,SSRDB)遥感图像时空融合算法.该算法具有如下3个特点:(1)构建了互补性的网络框架,虽然该框架是端到端的深度学习模型,但每个模块有各自的物理意义和任务,通过增加中间监督,分别实现空间维的超分建模,时间维的变化预测建模,以及两者优势互补的融合建模;(2)对变化预测的数学表示进行推演,利用一个非线性补偿模块,使得两分支共享超分模块,在共享超分模块和递归复用超分单元的双重策略下,显著降低了网络参数;(3)递归超分模块使用固定的2倍率超分单元,在有效监督和有效参考下,渐进式进行特征增强与图像重建,这可以有效提高超分精度,且通过调整超分单元个数,灵活适应不同倍率差异的时空融合任务.SSRDB算法在空间和光谱特性上以及变化区域上都展现了优秀的融合效果,RMSE(Root Mean Squared Error)、SAM(Spectral Angle Mapper)和SSIM(Structural Similarity)3个定量评价指标显示,在CIA(Coleambally lrrigation Area)数据集上分别优于次优方法 7.067%、2.065%、0.563%;在LGC(Lower Gwydir Catchment)数据集上分别优于次优方法5.319%、5.490%、1.455%;在Nanjing数据集上分别优于次优方法6.486%、16.290%、0.481%.
In this paper,we analyze the fusion law of scene weak change region and type change region,the difference of physical model and the complementarity of effect from spatial and temporal dimensions,and propose a shared superresolution dual-branch(Shared Super-Resolution Dual-Branch,SSRDB) remote sensing image spatio-temporal fusion algorithm.The algorithm has the following three characteristics:(1) A complementary network framework is constructed.Although the framework is an end-to-end deep learning model,each module has its own physical meaning and task.By adding intermediate supervision,the super-resolution modeling of spatial dimension,the change prediction modeling of time dimension and the fusion modeling of the two advantages are realized respectively.(2) The mathematical representation of the change prediction is deduced,and a nonlinear compensation module is used to make the two branches share the super-resolution module.Under the dual strategy of sharing super-resolution module and recursive multiplexing super-resolution unit,the network parameters are significantly reduced.(3) The recursive super-resolution module uses fixed 2-magnification super-resolution units to gradually enhance features and reconstruct images under effective supervision and reference,which can effectively improve the precision of super-resolution,and flexibly adapt to spatio-temporal fusion tasks with different magnification differences by adjusting the number of super-resolution units.The SSRDB algorithm shows excellent fusion effect in spatial and spectral characteristics and change regions.The three quantitative evaluation indexes of RMSE(Root Mean Squared Error)、SAM(Spectral Angle Mapper) and SSIM(Structural Similarity) show that it is superior to the suboptimal method on the CIA(Coleambally lrrigation Area) dataset by 7.067%,2.065% and 0.563%,respectively.On the LGC(Lower Gwydir Catchment) dataset,it is superior to the sub-optimal method by 5.319%,5.490% and 1.455%,respectively.On the Nanjing dataset,it is superior to the suboptimal method by 6.486%,16.290% and 0.481%,respectively.
作者
方帅
张小溪
张晶
FANG Shuai;ZHANG Xiao-xi;ZHANG Jing(School of Computer and Information,Hefei University of Technology,Hefei,Anhui 230000,China;Anhui Province Key Laboratory of Industry Safety and Emergency Technology,Hefei,Anhui 230000,China)
出处
《电子学报》
北大核心
2025年第2期581-594,共14页
Acta Electronica Sinica
基金
国家自然科学基金(No.61872327,No.61175033)。
关键词
遥感图像
时空融合
双分支
图像超分
卷积神经网络
remote sensing image
spatiotemporal fusion
dual-branch
image super-resolution
convolutional neural network
作者简介
方帅,女,1978年1月出生于安徽省淮南市.现为合肥工业大学计算机与信息学院教授、博士生导师.主要研究方向为图像复原和视觉检测.E-mail:fangshuai@hfut.edu.cn;张小溪,女,2001年12月出生于河南省周口市.现为合肥工业大学计算机与信息学院硕士研究生.主要研究方向为图像复原和视觉检测.E-mail:2022171263@mail.hfut.edu.cn;张晶,女,1976年10月出生于安徽省合肥市.现为合肥工业大学计算机与信息学院副教授.主要研究方向为人工智能、数据挖掘.E-mail:zhangjing@hfut.edu.cn。
