摘要
目的目前的去雾算法已能够较好地处理均质的薄雾图像,但针对雾霾浓度不同的非均质雾霾图像往往具有较低的去雾性能。为此,提出了结合浓度划分与图像融合的多分支非均质图像去雾算法。方法本文将单幅非均质雾霾图像视为由多个具有均质薄雾或者均质浓雾的局部区域组成,通过分别解决单幅非均质雾图中的不同均质雾霾区域来进行整幅非均质图像去雾。首先在不同均质雾霾浓度的去雾数据集上训练了多个图像增强网络,以得到针对不同均质雾霾浓度的图像增强模型,它们对于相应雾霾浓度的图像区域具有较好的增强性能。由于单个图像增强模型只能较好地增强一幅非均质雾霾图像中具有对应雾霾浓度的图像区域,但对其他不同雾霾浓度的图像区域可能存在去雾力度不足或者过度增强的现象,本文又设计了一个图像融合网络,将多个初始图像增强结果中的优势区域进行融合,得到最终的图像去雾结果。结果大量的实验结果显示,在合成雾霾数据集FiveK-Haze上,本文算法与排名第2的SCAN(self-paced semi-curricular attention network)方法相比在峰值信噪比(peak signal-tonoise ratio,PSNR)和结构相似性(structural similarity index,SSIM)有参考指标上分别提高了5.2866 dB和0.1138。在真实雾霾数据集Real-World上,本文算法与排名第2的DEAN(detail-enhanced convolution and content-guided attention network)方法相比,在FADE(fog aware density evaluator)和HazDes无参考指标上分别降低了0.0793和0.0512。在室内合成测试数据集SOTS-indoor(synthetic objective testing set)上,本文算法的PSNR和SSIM指标比排名第2的DeFormer方法分别提高了2.5182 dB和0.0123。在室外合成测试数据集SOTS-outdoor上,本文算法在PSNR指标上比排名第2的SGID-PFF(self-guided image dehazing using progressive feature fusion)方法提高了2.832 dB,在SSIM指标上比排名第2的DeFormer方法提高了0.0238。结论与已有的单幅图像去雾方法相比,本文算法能够有效增强非均质雾霾图像,具有更高的鲁棒性,展现出较好的性能指标。
Objective When capturing images using a camera,atmospheric floating particles,such as smoke,dust,and fog,can affect image quality,leading to decreased clarity.These compromised images not only increase the likelihood of human visual misjudgment but also hinder the development of visual tasks such as remote sensing monitoring and autonomous driving.Current dehazing methods are effective for homogeneous thin hazy images but often perform poorly on the nonhomogeneous hazy images.Therefore,a multibranch non-homogeneous image dehazing method combined with concentration partitioning and image fusion is proposed to address these challenges.A single non-homogeneous hazy image is regarded as a combination of multiple local regions with homogeneous thin or dense haze.The entire non-homogeneous image is dehazed by separately addressing different homogeneous hazy regions in a single nonhomogeneous hazy image.Method Concentration partitioning and image fusion based multi-branch image dehazing neural network(CPIFNet),a twostage network framework for image enhancement and image fusion,is then designed.Experiment results revealed that training models based on homogeneous haze image datasets with different haze concentrations can lead to enhancement in image models with varying enhancement intensities.Homogeneous hazed image datasets with different haze concentrations are necessary to obtain varying enhancement models.FiveK-Haze is a synthesized dehazing dataset based on the atmospheric physical scattering model,encompassing nine types of different homogeneous hazed images with varying haze concentrations.The hazy images in the FiveK-Haze dataset are re-partitioned based on haze concentration,dividing the dataset into 1 to 5 different haze concentration levels to exclude the hazy image samples with excessive haze concentrations.Then,the image enhancement network is trained on those new homogeneous dehazing datasets to obtain image enhancement models for different haze concentrations.In the image enhancement networks,the deep image features of hazy images are continuously extracted to obtain the stretching coefficient of the image enhancement model.This coefficient is multiplied with the hazy image to produce the image enhancement result.The image enhancement network replaces network layers with residual modules to extract deep feature information,avoiding feature information loss as the network deepens.The ReLU activation function is used after each convolutional layer to accelerate the convergence speed of network training and avoid the transmission of negative values in the feature layers.Each enhancement network performs well for the corresponding haze concentration image region.However,a single enhancement network can only effectively enhance image regions with corresponding haze concentrations,which leads to insufficient or excessive dehazing in other regions.Therefore,an image fusion network is designed to combine the advantageous regions in the multiple initial enhancement results,producing the final dehazed result.In the image fusion network,deep image features of different image enhancement results are continuously extracted,and the dehazed result is obtained by stacking and merging these deep image features.In addition to reconstruction,perceptual,and structural losses,the image enhancement and image fusion networks also utilize color loss to constrain the image dehazing results of the network modules.This condition is due to the severe loss of pixel information in dense hazy images,complicating color restoration and allowing the color loss function to guide the color of the image dehazing result closer to the reference image.Result Theoretically,a dehazing dataset with more finely divided haze concentration levels could enable network models to learn additional information of hazed images.However,experiments reveal otherwise.When the number of haze datasets is 3,that is,the number of image enhancement models is 3,CPIFNet achieves the best dehazing performance.Large-scale experiments are conducted in comparison with more than 10 latest image dehazing algorithms,revealing that the proposed method is optimal in terms of performance indicators and dehazing effects.Compared with the second-ranked self-paced semi-curricular attention network(SCAN)method,the method improves the reference indicators of peak signal-to-noise ratio(PSNR)and structural similarity index(SSIM)by 5.2866 dB and 0.1138,respectively,compared with the synthetic hazy image dataset FiveK-Haze.Compared with the second-ranked DEAN method,the proposed method reduces the non-reference metrics as FADE and HazDes by 0.0793 and 0.0512,respectively,over the real-world hazy image dataset.Additionally,additional tests are conducted on some publicly available datasets to obtain more comparative experiments and indicator evaluations.On the SOTS-indoor dataset,the method improves PSNR and SSIM by 2.5182 dB and 0.0123,respectively,compared to the second-ranked DeFormer method.On the SOTSoutdoor dataset,the method improves PSNR by 2.832 dB compared to the second-ranked SGID-PFF method and enhances SSIM by 0.0238 compared to the second-ranked DeFormer method.Conclusion A two-stage,multi-branch deep neural network is designed to remove haze from a single non-homogeneous hazy image by separately addressing different homogeneous hazy regions.Compared to existing methods,the proposed method can enhance the structural contrast of dense hazy regions and slightly improve thin hazy regions while restoring color information.
作者
金鑫乐
刘春晓
叶爽爽
王成骅
周子翔
Jin Xinle;Liu Chunxiao;Ye Shuangshuang;Wang Chenghua;Zhou Zixiang(School of Computer Science and Technology,Zhejiang Gongshang University,Hangzhou 310018,China)
出处
《中国图象图形学报》
北大核心
2025年第3期798-810,共13页
Journal of Image and Graphics
基金
国家自然科学基金项目(61976188)
浙江省自然科学基金项目(LY24F020004,LZ23F020004)
浙江省重点研发计划资助(2023C01039)
国家级大学生创新创业训练计划项目(GJ202313014)
浙江工商大学“数字+”学科建设项目(SZJ2022B016)
浙江省大学生科技创新活动计划暨新苗人才计划项目(2023R408035,2023R408072)。
关键词
图像去雾
非均质雾霾图像
雾霾浓度划分
图像融合
多分支神经网络
image dehazing
non-homogeneous hazy images
haze concentration partitioning
image fusion
multi-branch neural network
作者简介
金鑫乐,男,本科生,主要研究方向为图像增强与复原技术E-mail:2112190125@pop.zjgsu.edu.cn;通信作者:刘春晓,男,副教授,主要研究方向为计算机视觉与计算机图形学、机器学习与智能系统。E-mail:cxliu@mail.zjgsu.edu.cn;叶爽爽,女,硕士研究生,主要研究方向为机器学习与计算机视觉。E-mail:20020100039@pop.zjgsu.edu.cn;王成骅,男,本科生,主要研究方向为机器学习与计算机视觉。E-mail:2012190218@pop.zjgsu.edu.cn;周子翔,男,本科生,主要研究方向为机器学习与计算机视觉。E-mail:2011080223@pop.zjgsu.edu.cn。
