Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices...Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.展开更多
随着钓鱼邮件数量的迅速增加以及对抗技术的不断演进,传统的钓鱼邮件检测方法在效率和准确性方面面临严峻挑战.为此,提出了一种基于大语言模型(large language model,LLM)的钓鱼邮件检测方法,以解决现有系统检测率低、漏报率高及人机交...随着钓鱼邮件数量的迅速增加以及对抗技术的不断演进,传统的钓鱼邮件检测方法在效率和准确性方面面临严峻挑战.为此,提出了一种基于大语言模型(large language model,LLM)的钓鱼邮件检测方法,以解决现有系统检测率低、漏报率高及人机交互性差等问题.通过全面分析钓鱼邮件的关键特征,包括邮件头部字段、正文内容、URL、二维码、附件及HTML页面,利用特征插入算法构建高质量的训练数据集.基于预训练语言模型LLaMA和低秩自适应微调技术(low-rank adaptation,LoRA),在仅更新0.72%模型参数(约50 MB)条件下实现领域知识迁移,获得钓鱼邮件检测大模型.实验结果显示,与传统方法相比,基于大语言模型的检测方法显著提升了检测的准确性与鲁棒性,整体准确率达到94.5%,有效降低了误报率,增强了钓鱼邮件特征的分类与解释能力,提供了更具实用性和可靠性的钓鱼邮件检测方案.展开更多
基金supported by the National Natural Science Foundation of China(62171088,U19A2052,62020106011)the Medico-Engineering Cooperation Funds from University of Electronic Science and Technology of China(ZYGX2021YGLH215,ZYGX2022YGRH005)。
文摘Deep neural networks(DNNs)have achieved great success in many data processing applications.However,high computational complexity and storage cost make deep learning difficult to be used on resource-constrained devices,and it is not environmental-friendly with much power cost.In this paper,we focus on low-rank optimization for efficient deep learning techniques.In the space domain,DNNs are compressed by low rank approximation of the network parameters,which directly reduces the storage requirement with a smaller number of network parameters.In the time domain,the network parameters can be trained in a few subspaces,which enables efficient training for fast convergence.The model compression in the spatial domain is summarized into three categories as pre-train,pre-set,and compression-aware methods,respectively.With a series of integrable techniques discussed,such as sparse pruning,quantization,and entropy coding,we can ensemble them in an integration framework with lower computational complexity and storage.In addition to summary of recent technical advances,we have two findings for motivating future works.One is that the effective rank,derived from the Shannon entropy of the normalized singular values,outperforms other conventional sparse measures such as the?_1 norm for network compression.The other is a spatial and temporal balance for tensorized neural networks.For accelerating the training of tensorized neural networks,it is crucial to leverage redundancy for both model compression and subspace training.
文摘随着钓鱼邮件数量的迅速增加以及对抗技术的不断演进,传统的钓鱼邮件检测方法在效率和准确性方面面临严峻挑战.为此,提出了一种基于大语言模型(large language model,LLM)的钓鱼邮件检测方法,以解决现有系统检测率低、漏报率高及人机交互性差等问题.通过全面分析钓鱼邮件的关键特征,包括邮件头部字段、正文内容、URL、二维码、附件及HTML页面,利用特征插入算法构建高质量的训练数据集.基于预训练语言模型LLaMA和低秩自适应微调技术(low-rank adaptation,LoRA),在仅更新0.72%模型参数(约50 MB)条件下实现领域知识迁移,获得钓鱼邮件检测大模型.实验结果显示,与传统方法相比,基于大语言模型的检测方法显著提升了检测的准确性与鲁棒性,整体准确率达到94.5%,有效降低了误报率,增强了钓鱼邮件特征的分类与解释能力,提供了更具实用性和可靠性的钓鱼邮件检测方案.
