The rapid integration of Internet of Things(IoT)technologies is reshaping the global energy landscape by deploying smart meters that enable high-resolution consumption monitoring,two-way communication,and advanced met...The rapid integration of Internet of Things(IoT)technologies is reshaping the global energy landscape by deploying smart meters that enable high-resolution consumption monitoring,two-way communication,and advanced metering infrastructure services.However,this digital transformation also exposes power system to evolving threats,ranging from cyber intrusions and electricity theft to device malfunctions,and the unpredictable nature of these anomalies,coupled with the scarcity of labeled fault data,makes realtime detection exceptionally challenging.To address these difficulties,a real-time decision support framework is presented for smart meter anomality detection that leverages rolling time windows and two self-supervised contrastive learning modules.The first module synthesizes diverse negative samples to overcome the lack of labeled anomalies,while the second captures intrinsic temporal patterns for enhanced contextual discrimination.The end-to-end framework continuously updates its model with rolling updated meter data to deliver timely identification of emerging abnormal behaviors in evolving grids.Extensive evaluations on eight publicly available smart meter datasets over seven diverse abnormal patterns testing demonstrate the effectiveness of the proposed full framework,achieving average recall and F1 score of more than 0.85.展开更多
Federated learning(FL)is a distributed machine learning paradigm for edge cloud computing.FL can facilitate data-driven decision-making in tactical scenarios,effectively addressing both data volume and infrastructure ...Federated learning(FL)is a distributed machine learning paradigm for edge cloud computing.FL can facilitate data-driven decision-making in tactical scenarios,effectively addressing both data volume and infrastructure challenges in edge environments.However,the diversity of clients in edge cloud computing presents significant challenges for FL.Personalized federated learning(pFL)received considerable attention in recent years.One example of pFL involves exploiting the global and local information in the local model.Current pFL algorithms experience limitations such as slow convergence speed,catastrophic forgetting,and poor performance in complex tasks,which still have significant shortcomings compared to the centralized learning.To achieve high pFL performance,we propose FedCLCC:Federated Contrastive Learning and Conditional Computing.The core of FedCLCC is the use of contrastive learning and conditional computing.Contrastive learning determines the feature representation similarity to adjust the local model.Conditional computing separates the global and local information and feeds it to their corresponding heads for global and local handling.Our comprehensive experiments demonstrate that FedCLCC outperforms other state-of-the-art FL algorithms.展开更多
Heterogeneous federated learning(HtFL)has gained significant attention due to its ability to accommodate diverse models and data from distributed combat units.The prototype-based HtFL methods were proposed to reduce t...Heterogeneous federated learning(HtFL)has gained significant attention due to its ability to accommodate diverse models and data from distributed combat units.The prototype-based HtFL methods were proposed to reduce the high communication cost of transmitting model parameters.These methods allow for the sharing of only class representatives between heterogeneous clients while maintaining privacy.However,existing prototype learning approaches fail to take the data distribution of clients into consideration,which results in suboptimal global prototype learning and insufficient client model personalization capabilities.To address these issues,we propose a fair trainable prototype federated learning(FedFTP)algorithm,which employs a fair sampling training prototype(FSTP)mechanism and a hyperbolic space constraints(HSC)mechanism to enhance the fairness and effectiveness of prototype learning on the server in heterogeneous environments.Furthermore,a local prototype stable update(LPSU)mechanism is proposed as a means of maintaining personalization while promoting global consistency,based on contrastive learning.Comprehensive experimental results demonstrate that FedFTP achieves state-of-the-art performance in HtFL scenarios.展开更多
The encoding aperture snapshot spectral imaging system,based on the compressive sensing theory,can be regarded as an encoder,which can efficiently obtain compressed two-dimensional spectral data and then decode it int...The encoding aperture snapshot spectral imaging system,based on the compressive sensing theory,can be regarded as an encoder,which can efficiently obtain compressed two-dimensional spectral data and then decode it into three-dimensional spectral data through deep neural networks.However,training the deep neural net⁃works requires a large amount of clean data that is difficult to obtain.To address the problem of insufficient training data for deep neural networks,a self-supervised hyperspectral denoising neural network based on neighbor⁃hood sampling is proposed.This network is integrated into a deep plug-and-play framework to achieve self-supervised spectral reconstruction.The study also examines the impact of different noise degradation models on the fi⁃nal reconstruction quality.Experimental results demonstrate that the self-supervised learning method enhances the average peak signal-to-noise ratio by 1.18 dB and improves the structural similarity by 0.009 compared with the supervised learning method.Additionally,it achieves better visual reconstruction results.展开更多
输电线路巡检中采集的螺栓图像有分辨率低、视觉信息不足的特点。针对传统图像分类模型难以从螺栓图像中学习到语义丰富的视觉表征问题,提出了一种基于多模态对比学习的输电线路螺栓缺陷分类方法。首先,为了将文本中螺栓相关的语义信息...输电线路巡检中采集的螺栓图像有分辨率低、视觉信息不足的特点。针对传统图像分类模型难以从螺栓图像中学习到语义丰富的视觉表征问题,提出了一种基于多模态对比学习的输电线路螺栓缺陷分类方法。首先,为了将文本中螺栓相关的语义信息和先验知识以跨模态的方式注入视觉表征,提出了一种结合多模态对比预训练和监督式微调的二阶段训练算法;其次,为了缓解多模态对比预训练中的过拟合问题,提出了标签平滑的信息噪声对比估计损失(info noise contrastive estimation loss with label smoothing,infoNCE-LS),以提高预训练视觉表征的泛化性能;最后,针对上下游任务的不匹配问题,设计了3种基于文本提示的分类头,以改善预训练视觉表征在监督式微调阶段的迁移学习效果。实验结果表明:该文基于Res Net50和ViT构建的两种模型在螺栓缺陷分类数据集上的准确率分别为92.3%和97.4%,相比基线分别提高了2.4%和5.8%。研究实现了从文本到图像的语义信息跨模态补充,为螺栓缺陷识别的研究提供了新的思路。展开更多
社交媒体上图像和文本数据的快速增长导致人们对多模态讽刺检测问题的关注不断提高。然而,现有基于特征提取融合的检测方法存在一些缺陷:一是大多数方法缺乏多模态检测所需的底层模态对齐能力,二是模态融合过程忽视了模态间的动态关系,...社交媒体上图像和文本数据的快速增长导致人们对多模态讽刺检测问题的关注不断提高。然而,现有基于特征提取融合的检测方法存在一些缺陷:一是大多数方法缺乏多模态检测所需的底层模态对齐能力,二是模态融合过程忽视了模态间的动态关系,三是未能充分利用模态互补性。为此,提出一种基于单模态监督对比学习、多模态融合和多视图聚合预测的检测模型。以CLIP(contrastive language image pre-training)模型作为编码器来增强图像和文本底层编码的对齐效果。结合单模态监督对比学习方法,通过单模态预测来指导模态间的动态关系。然后,设计了全局-局部跨模态融合方法,利用每种模态的语义级表示作为全局多模态上下文与局部单模态特征进行交互,通过多个跨模态融合层提高模态融合效果,并减少了以往局部-局部跨模态融合方法的时间和空间成本。采用多视图聚合预测方法充分利用图像、文本和图文视图的互补性。总之,该模型能有效捕捉多模态讽刺数据的跨模态语义不一致性,在公开数据集MSD上取得了比现有最好方法DMSD-Cl更好的结果。展开更多
命名实体识别任务旨在识别出非结构化文本中所包含的实体并将其分配给预定义的实体类别中.随着互联网和社交媒体的发展,文本信息往往伴随着图像等视觉模态信息出现,传统的命名实体识别方法在多模态信息中表现不佳.近年来,多模态命名实...命名实体识别任务旨在识别出非结构化文本中所包含的实体并将其分配给预定义的实体类别中.随着互联网和社交媒体的发展,文本信息往往伴随着图像等视觉模态信息出现,传统的命名实体识别方法在多模态信息中表现不佳.近年来,多模态命名实体识别任务广受重视.然而,现有的多模态命名实体识别方法中,存在跨模态知识间的细粒度对齐不足问题,文本表征会融合语义不相关的图像信息,进而引入噪声.为了解决这些问题,提出了一种基于细粒度图文对齐的多模态命名实体识别方法(FGITA:A Multi-Modal NER Frame based on Fine-Grained Image-Text Alignment).首先,该方法通过目标检测、语义相似性判断等,确定更为细粒度的文本实体和图像子对象之间的语义相关性;其次,通过双线性注意力机制,计算出图像子对象与实体的相关性权重,并依据权重将子对象信息融入到实体表征中;最后,提出了一种跨模态对比学习方法,依据图像和实体之间的匹配程度,优化实体和图像在嵌入空间中的距离,借此帮助实体表征学习相关的图像信息.在两个公开数据集上的实验表明,FGITA优于5个主流多模态命名实体识别方法,验证了方法的有效性,同时验证了细粒度跨模态对齐在多模态命名实体识别任务中的重要性和优越性.展开更多
输电线路的智能巡检视觉任务对电力系统的安全稳定至关重要。尽管深度学习网络在分布一致的训练和测试数据集上表现良好,但实际应用中数据分布的偏差常常会降低模型性能。为了解决这一问题,提出一种基于对比学习的训练方法(TMCL),旨在...输电线路的智能巡检视觉任务对电力系统的安全稳定至关重要。尽管深度学习网络在分布一致的训练和测试数据集上表现良好,但实际应用中数据分布的偏差常常会降低模型性能。为了解决这一问题,提出一种基于对比学习的训练方法(TMCL),旨在增强模型鲁棒性。首先,构建专为输电线路场景设计的基准测试集TLD-C(Transmission Line Dataset-Corruption)用于评估模型在面对图像损坏时的鲁棒性;其次,通过构建对类别特征敏感的正负样本对,提升模型对不同类别特征的区分能力;然后,使用结合对比损失和交叉熵损失的联合优化策略对特征提取过程施加额外约束,以优化特征向量的表征;最后,引入非局部特征去噪网络(NFD)用于提取与类别密切相关的特征。实验结果表明,模型改进后的训练方法在输电线路数据集(TLD)上的平均精度比原始方法高出3.40个百分点,在TLD-C数据集上的相对损坏精度(rCP)比原始方法高出4.69个百分点。展开更多
文摘The rapid integration of Internet of Things(IoT)technologies is reshaping the global energy landscape by deploying smart meters that enable high-resolution consumption monitoring,two-way communication,and advanced metering infrastructure services.However,this digital transformation also exposes power system to evolving threats,ranging from cyber intrusions and electricity theft to device malfunctions,and the unpredictable nature of these anomalies,coupled with the scarcity of labeled fault data,makes realtime detection exceptionally challenging.To address these difficulties,a real-time decision support framework is presented for smart meter anomality detection that leverages rolling time windows and two self-supervised contrastive learning modules.The first module synthesizes diverse negative samples to overcome the lack of labeled anomalies,while the second captures intrinsic temporal patterns for enhanced contextual discrimination.The end-to-end framework continuously updates its model with rolling updated meter data to deliver timely identification of emerging abnormal behaviors in evolving grids.Extensive evaluations on eight publicly available smart meter datasets over seven diverse abnormal patterns testing demonstrate the effectiveness of the proposed full framework,achieving average recall and F1 score of more than 0.85.
基金supported by the Natural Science Foundation of Xinjiang Uygur Autonomous Region(Grant No.2022D01B 187)。
文摘Federated learning(FL)is a distributed machine learning paradigm for edge cloud computing.FL can facilitate data-driven decision-making in tactical scenarios,effectively addressing both data volume and infrastructure challenges in edge environments.However,the diversity of clients in edge cloud computing presents significant challenges for FL.Personalized federated learning(pFL)received considerable attention in recent years.One example of pFL involves exploiting the global and local information in the local model.Current pFL algorithms experience limitations such as slow convergence speed,catastrophic forgetting,and poor performance in complex tasks,which still have significant shortcomings compared to the centralized learning.To achieve high pFL performance,we propose FedCLCC:Federated Contrastive Learning and Conditional Computing.The core of FedCLCC is the use of contrastive learning and conditional computing.Contrastive learning determines the feature representation similarity to adjust the local model.Conditional computing separates the global and local information and feeds it to their corresponding heads for global and local handling.Our comprehensive experiments demonstrate that FedCLCC outperforms other state-of-the-art FL algorithms.
基金supported by the Natural Science Foundation of Xinjiang Uygur Autonomous Region(No.2022D01B187).
文摘Heterogeneous federated learning(HtFL)has gained significant attention due to its ability to accommodate diverse models and data from distributed combat units.The prototype-based HtFL methods were proposed to reduce the high communication cost of transmitting model parameters.These methods allow for the sharing of only class representatives between heterogeneous clients while maintaining privacy.However,existing prototype learning approaches fail to take the data distribution of clients into consideration,which results in suboptimal global prototype learning and insufficient client model personalization capabilities.To address these issues,we propose a fair trainable prototype federated learning(FedFTP)algorithm,which employs a fair sampling training prototype(FSTP)mechanism and a hyperbolic space constraints(HSC)mechanism to enhance the fairness and effectiveness of prototype learning on the server in heterogeneous environments.Furthermore,a local prototype stable update(LPSU)mechanism is proposed as a means of maintaining personalization while promoting global consistency,based on contrastive learning.Comprehensive experimental results demonstrate that FedFTP achieves state-of-the-art performance in HtFL scenarios.
基金Supported by the Zhejiang Provincial"Jianbing"and"Lingyan"R&D Programs(2023C03012,2024C01126)。
文摘The encoding aperture snapshot spectral imaging system,based on the compressive sensing theory,can be regarded as an encoder,which can efficiently obtain compressed two-dimensional spectral data and then decode it into three-dimensional spectral data through deep neural networks.However,training the deep neural net⁃works requires a large amount of clean data that is difficult to obtain.To address the problem of insufficient training data for deep neural networks,a self-supervised hyperspectral denoising neural network based on neighbor⁃hood sampling is proposed.This network is integrated into a deep plug-and-play framework to achieve self-supervised spectral reconstruction.The study also examines the impact of different noise degradation models on the fi⁃nal reconstruction quality.Experimental results demonstrate that the self-supervised learning method enhances the average peak signal-to-noise ratio by 1.18 dB and improves the structural similarity by 0.009 compared with the supervised learning method.Additionally,it achieves better visual reconstruction results.
文摘输电线路巡检中采集的螺栓图像有分辨率低、视觉信息不足的特点。针对传统图像分类模型难以从螺栓图像中学习到语义丰富的视觉表征问题,提出了一种基于多模态对比学习的输电线路螺栓缺陷分类方法。首先,为了将文本中螺栓相关的语义信息和先验知识以跨模态的方式注入视觉表征,提出了一种结合多模态对比预训练和监督式微调的二阶段训练算法;其次,为了缓解多模态对比预训练中的过拟合问题,提出了标签平滑的信息噪声对比估计损失(info noise contrastive estimation loss with label smoothing,infoNCE-LS),以提高预训练视觉表征的泛化性能;最后,针对上下游任务的不匹配问题,设计了3种基于文本提示的分类头,以改善预训练视觉表征在监督式微调阶段的迁移学习效果。实验结果表明:该文基于Res Net50和ViT构建的两种模型在螺栓缺陷分类数据集上的准确率分别为92.3%和97.4%,相比基线分别提高了2.4%和5.8%。研究实现了从文本到图像的语义信息跨模态补充,为螺栓缺陷识别的研究提供了新的思路。
文摘社交媒体上图像和文本数据的快速增长导致人们对多模态讽刺检测问题的关注不断提高。然而,现有基于特征提取融合的检测方法存在一些缺陷:一是大多数方法缺乏多模态检测所需的底层模态对齐能力,二是模态融合过程忽视了模态间的动态关系,三是未能充分利用模态互补性。为此,提出一种基于单模态监督对比学习、多模态融合和多视图聚合预测的检测模型。以CLIP(contrastive language image pre-training)模型作为编码器来增强图像和文本底层编码的对齐效果。结合单模态监督对比学习方法,通过单模态预测来指导模态间的动态关系。然后,设计了全局-局部跨模态融合方法,利用每种模态的语义级表示作为全局多模态上下文与局部单模态特征进行交互,通过多个跨模态融合层提高模态融合效果,并减少了以往局部-局部跨模态融合方法的时间和空间成本。采用多视图聚合预测方法充分利用图像、文本和图文视图的互补性。总之,该模型能有效捕捉多模态讽刺数据的跨模态语义不一致性,在公开数据集MSD上取得了比现有最好方法DMSD-Cl更好的结果。
文摘命名实体识别任务旨在识别出非结构化文本中所包含的实体并将其分配给预定义的实体类别中.随着互联网和社交媒体的发展,文本信息往往伴随着图像等视觉模态信息出现,传统的命名实体识别方法在多模态信息中表现不佳.近年来,多模态命名实体识别任务广受重视.然而,现有的多模态命名实体识别方法中,存在跨模态知识间的细粒度对齐不足问题,文本表征会融合语义不相关的图像信息,进而引入噪声.为了解决这些问题,提出了一种基于细粒度图文对齐的多模态命名实体识别方法(FGITA:A Multi-Modal NER Frame based on Fine-Grained Image-Text Alignment).首先,该方法通过目标检测、语义相似性判断等,确定更为细粒度的文本实体和图像子对象之间的语义相关性;其次,通过双线性注意力机制,计算出图像子对象与实体的相关性权重,并依据权重将子对象信息融入到实体表征中;最后,提出了一种跨模态对比学习方法,依据图像和实体之间的匹配程度,优化实体和图像在嵌入空间中的距离,借此帮助实体表征学习相关的图像信息.在两个公开数据集上的实验表明,FGITA优于5个主流多模态命名实体识别方法,验证了方法的有效性,同时验证了细粒度跨模态对齐在多模态命名实体识别任务中的重要性和优越性.
文摘输电线路的智能巡检视觉任务对电力系统的安全稳定至关重要。尽管深度学习网络在分布一致的训练和测试数据集上表现良好,但实际应用中数据分布的偏差常常会降低模型性能。为了解决这一问题,提出一种基于对比学习的训练方法(TMCL),旨在增强模型鲁棒性。首先,构建专为输电线路场景设计的基准测试集TLD-C(Transmission Line Dataset-Corruption)用于评估模型在面对图像损坏时的鲁棒性;其次,通过构建对类别特征敏感的正负样本对,提升模型对不同类别特征的区分能力;然后,使用结合对比损失和交叉熵损失的联合优化策略对特征提取过程施加额外约束,以优化特征向量的表征;最后,引入非局部特征去噪网络(NFD)用于提取与类别密切相关的特征。实验结果表明,模型改进后的训练方法在输电线路数据集(TLD)上的平均精度比原始方法高出3.40个百分点,在TLD-C数据集上的相对损坏精度(rCP)比原始方法高出4.69个百分点。
