摘要
针对成对数据集获取成本较高、光照分布不均衡图像增强效果欠佳以及增强结果易产生十字形伪影的问题,提出了一种基于生成对抗网络结合Transformer的半监督图像增强方法。首先,采用Transformer网络架构作为生成对抗网络中生成器的主干网络,提取不同像素块间的依赖关系以获取全局特征,并通过非成对数据集进行半监督学习;其次,使用灰度图作为生成器网络的光照注意力图,以平衡增强结果在不同区域的曝光水平;最后,在生成器和鉴别器网络中交叉使用均等裁剪策略和滑动窗口裁剪策略,增强网络提取特征的能力并解决十字形伪影问题,并引入重建损失来提高生成器对图像细节的感知能力。结果表明:提出方法取得了更好的光照和色彩平衡效果,自然图像质量评估指标平均提升了2.37%;在图像修饰任务中,图像峰值信噪比、相似结构度和感知损失同时达到了最优;在低光照增强任务中,图像峰值信噪比提升了13.46%;充分验证了提出方法在图像增强2个子任务上的有效性。
To address the issues of high cost in acquiring paired datasets,inadequate enhancement effects due to uneven lighting distributions,and the occurrence of cross-shaped artifacts in the enhanced results,a semi-supervised image enhancement method based on the combination of generative adversarial network and Transformer was proposed.Firstly,the Transformer network architecture was employed as the backbone network of the generator in the GAN to extract the dependency relationships between different pixel blocks for obtaining global features,and semi-supervised learning was performed using non-paired datasets.Secondly,a grayscale image was used as the illumination attention map for the generator network to balance the exposure levels of the enhanced results in different regions.Finally,equal cropping strategy and sliding window cropping strategy were cross-used in the generator and discriminator networks to enhance the feature extraction capability of the network and solve the problem of cross-shaped artifacts.Additionally,a reconstruction loss was introduced to improve the generator’s perception capability of image details.The results demonstrate that the proposed method has achieved better lighting and color balance effects,with an average improvement of 2.37%in the evaluation of natural image quality.In the image modification task,the peak signal-to-noise ratio,structural similarity,and perceptual loss simultaneously reach their optimum values.In the low-light enhancement task,the peak signal-to-noise ratio is improved by 13.46%.These results fully validate the effectiveness of the proposed method in the two subtasks of image enhancement.
作者
马天
李凡卉
席润韬
安金鹏
杨嘉怡
张杰慧
MA Tian;LI Fanhui;XI Runtao;AN Jinpeng;YANGJiayi;ZHANG Jiehui(College of Computer Science and Engineering,Xi’an University of Science and Technology,Xi’an 710054,China;CCTEG Changzhou Research Institute,Changzhou 213015,China;Tiandi(Changzhou)Automation Co.,Ltd.,Changzhou 213015,China)
出处
《西安科技大学学报》
CAS
北大核心
2023年第6期1207-1218,共12页
Journal of Xi’an University of Science and Technology
基金
国家重点研发计划课题项目(2021YFB4000905)
国家自然科学基金项目(62101432,62102309)
陕西省自然科学基础研究计划项目(2022JM-508)。
作者简介
通信作者:马天,男,河南商丘人,博士,副教授,E-mail:matian@xust.edu.cn。
