With the development of sensors,the application of multi-source remote sensing data has been widely concerned.Since hyperspectral image(HSI)contains rich spectral information while light detection and ranging(LiDAR)da...With the development of sensors,the application of multi-source remote sensing data has been widely concerned.Since hyperspectral image(HSI)contains rich spectral information while light detection and ranging(LiDAR)data contains elevation information,joint use of them for ground object classification can yield positive results,especially by building deep networks.Fortu-nately,multi-scale deep networks allow to expand the receptive fields of convolution without causing the computational and training problems associated with simply adding more network layers.In this work,a multi-scale feature fusion network is proposed for the joint classification of HSI and LiDAR data.First,we design a multi-scale spatial feature extraction module with cross-channel connections,by which spatial information of HSI data and elevation information of LiDAR data are extracted and fused.In addition,a multi-scale spectral feature extraction module is employed to extract the multi-scale spectral features of HSI data.Finally,joint multi-scale features are obtained by weighting and concatenation operations and then fed into the classifier.To verify the effective-ness of the proposed network,experiments are carried out on the MUUFL Gulfport and Trento datasets.The experimental results demonstrate that the classification performance of the proposed method is superior to that of other state-of-the-art methods.展开更多
In this study,an underwater image enhancement method based on multi-scale adversarial network was proposed to solve the problem of detail blur and color distortion in underwater images.Firstly,the local features of ea...In this study,an underwater image enhancement method based on multi-scale adversarial network was proposed to solve the problem of detail blur and color distortion in underwater images.Firstly,the local features of each layer were enhanced into the global features by the proposed residual dense block,which ensured that the generated images retain more details.Secondly,a multi-scale structure was adopted to extract multi-scale semantic features of the original images.Finally,the features obtained from the dual channels were fused by an adaptive fusion module to further optimize the features.The discriminant network adopted the structure of the Markov discriminator.In addition,by constructing mean square error,structural similarity,and perceived color loss function,the generated image is consistent with the reference image in structure,color,and content.The experimental results showed that the enhanced underwater image deblurring effect of the proposed algorithm was good and the problem of underwater image color bias was effectively improved.In both subjective and objective evaluation indexes,the experimental results of the proposed algorithm are better than those of the comparison algorithm.展开更多
Offshore carbon dioxide(CO_(2)) geological storage(OCGS) represents a significant strategy for addressing climate change by curtailing greenhouse gas emissions. Nonetheless, the risk of CO_(2) leakage poses a substant...Offshore carbon dioxide(CO_(2)) geological storage(OCGS) represents a significant strategy for addressing climate change by curtailing greenhouse gas emissions. Nonetheless, the risk of CO_(2) leakage poses a substantial concern associated with this technology. This study introduces an innovative approach for establishing OCGS leakage scenarios, involving four pivotal stages, namely, interactive matrix establishment, risk matrix evaluation, cause–effect analysis, and scenario development, which has been implemented in the Pearl River Estuary Basin in China. The initial phase encompassed the establishment of an interaction matrix for OCGS systems based on features, events, and processes. Subsequent risk matrix evaluation and cause–effect analysis identified key system components, specifically CO_(2) injection and faults/features. Building upon this analysis, two leakage risk scenarios were successfully developed, accompanied by the corresponding mitigation measures. In addition, this study introduces the application of scenario development to risk assessment, including scenario numerical simulation and quantitative assessment. Overall, this research positively contributes to the sustainable development and safe operation of OCGS projects and holds potential for further refinement and broader application to diverse geographical environments and project requirements. This comprehensive study provides valuable insights into the establishment of OCGS leakage scenarios and demonstrates their practical application to risk assessment, laying the foundation for promoting the sustainable development and safe operation of ocean CO_(2) geological storage projects while proposing possibilities for future improvements and broader applications to different contexts.展开更多
Named entity recognition(NER)is an important part in knowledge extraction and one of the main tasks in constructing knowledge graphs.In today’s Chinese named entity recognition(CNER)task,the BERT-BiLSTM-CRF model is ...Named entity recognition(NER)is an important part in knowledge extraction and one of the main tasks in constructing knowledge graphs.In today’s Chinese named entity recognition(CNER)task,the BERT-BiLSTM-CRF model is widely used and often yields notable results.However,recognizing each entity with high accuracy remains challenging.Many entities do not appear as single words but as part of complex phrases,making it difficult to achieve accurate recognition using word embedding information alone because the intricate lexical structure often impacts the performance.To address this issue,we propose an improved Bidirectional Encoder Representations from Transformers(BERT)character word conditional random field(CRF)(BCWC)model.It incorporates a pre-trained word embedding model using the skip-gram with negative sampling(SGNS)method,alongside traditional BERT embeddings.By comparing datasets with different word segmentation tools,we obtain enhanced word embedding features for segmented data.These features are then processed using the multi-scale convolution and iterated dilated convolutional neural networks(IDCNNs)with varying expansion rates to capture features at multiple scales and extract diverse contextual information.Additionally,a multi-attention mechanism is employed to fuse word and character embeddings.Finally,CRFs are applied to learn sequence constraints and optimize entity label annotations.A series of experiments are conducted on three public datasets,demonstrating that the proposed method outperforms the recent advanced baselines.BCWC is capable to address the challenge of recognizing complex entities by combining character-level and word-level embedding information,thereby improving the accuracy of CNER.Such a model is potential to the applications of more precise knowledge extraction such as knowledge graph construction and information retrieval,particularly in domain-specific natural language processing tasks that require high entity recognition precision.展开更多
In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swa...In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.展开更多
Three materials(agar,konjac glucomannan(KGM)andκ-carrageenan)were used to prepare ternary systems,i.e.,sol-gels and their dried composites conditioned at varied relative humidity(RH)(33%,54%and 75%).Combined methods,...Three materials(agar,konjac glucomannan(KGM)andκ-carrageenan)were used to prepare ternary systems,i.e.,sol-gels and their dried composites conditioned at varied relative humidity(RH)(33%,54%and 75%).Combined methods,e.g.,scanning electron microscopy,small-angle X-ray scattering,infrared spectroscopy(IR)and X-ray diffraction(XRD),were used to disclose howκ-carrageenan addition tailors the features of agar/KGM/κ-carrageenan ternary system.As affirmed by IR and XRD,the ternary systems withκ-carrageenan below 25%(agar/KGM/carrageenan,50:25:25,m/m)displayed proper component interactions,which increased the sol-gel transition temperature and the hardness of obtained gels.For instance,the ternary composites could show hardness about 3 to 4 times higher than that for binary counterpart.These gels were dehydrated to acquire ternary composites.Compared to agar/KGM composite,the ternary composites showed fewer crystallites and nanoscale orders,and newly-formed nanoscale structures from chain assembly.Such multi-scale structures,for composites withκ-carrageenan below 25%,showed weaker changes with RH,as revealed by especially morphologic and crystalline features.Consequently,the ternary composites with lessκ-carrageenan(below 25%)exhibited stabilized elongation at break and hydrophilicity at different RHs.This hints to us that agar/KGM/κ-carrageenan composite systems can display series applications with improved features,e.g.,increased sol-gel transition point.展开更多
六自由度(Six Degrees of Freedom,6DoF)视频允许用户从全方位、任意视角身临其境体验场景,是下一代沉浸式视频产业的发展方向.部分自由度受限的窗口6DoF视频近年来成为研究热点,本文提出面向窗口6DoF合成视频的主观数据库和客观质量评...六自由度(Six Degrees of Freedom,6DoF)视频允许用户从全方位、任意视角身临其境体验场景,是下一代沉浸式视频产业的发展方向.部分自由度受限的窗口6DoF视频近年来成为研究热点,本文提出面向窗口6DoF合成视频的主观数据库和客观质量评价方法.在主观数据库方面,构建了包含两种交互路径不适性失真、四种绘制失真和四种压缩失真的窗口6DoF合成视频主观质量数据库Windowed-6DoF,并开展主观质量测试及结果分析.在客观质量评价方法方面,设计了一种融合多层特征的窗口6DoF合成视频无参考客观质量评价方法.采用切比雪夫矩提取视频时域切片上的底层形状特征;采用Resnet-50网络提取视频的时域、空域高层语义特征并进行降维处理;最后采用随机森林将底层形状特征和高层语义特征进行融合,且训练得到窗口6DoF合成视频的客观质量评价模型.在提出的数据库Windowed-6DoF和公共数据库IRCCyN/IVC DIBR的测试结果表明,本文提出的客观质量评价方法预测分数的皮尔逊线性相关系数分别达到0.9327和0.8581,与主观评价分数具有较好的一致性.展开更多
A large database is desired for machine learning(ML) technology to make accurate predictions of materials physicochemical properties based on their molecular structure.When a large database is not available,the develo...A large database is desired for machine learning(ML) technology to make accurate predictions of materials physicochemical properties based on their molecular structure.When a large database is not available,the development of proper featurization method based on physicochemical nature of target proprieties can improve the predictive power of ML models with a smaller database.In this work,we show that two new featurization methods,volume occupation spatial matrix and heat contribution spatial matrix,can improve the accuracy in predicting energetic materials' crystal density(ρ_(crystal)) and solid phase enthalpy of formation(H_(f,solid)) using a database containing 451 energetic molecules.Their mean absolute errors are reduced from 0.048 g/cm~3 and 24.67 kcal/mol to 0.035 g/cm~3 and 9.66 kcal/mol,respectively.By leave-one-out-cross-validation,the newly developed ML models can be used to determine the performance of most kinds of energetic materials except cubanes.Our ML models are applied to predict ρ_(crystal) and H_(f,solid) of CHON-based molecules of the 150 million sized PubChem database,and screened out 56 candidates with competitive detonation performance and reasonable chemical structures.With further improvement in future,spatial matrices have the potential of becoming multifunctional ML simulation tools that could provide even better predictions in wider fields of materials science.展开更多
针对红外小目标图像的低分辨率、特征信息少、识别准确率低等问题,提出嵌入空间位置信息和多视角特征提取(Embedded Spatial Location Information and Multi-view Feature Extraction,ESLIMFE)的红外小目标检测模型。首先,随着网络深...针对红外小目标图像的低分辨率、特征信息少、识别准确率低等问题,提出嵌入空间位置信息和多视角特征提取(Embedded Spatial Location Information and Multi-view Feature Extraction,ESLIMFE)的红外小目标检测模型。首先,随着网络深度的增加导致特征图分辨率逐渐减小从而丢失细节信息,因此在骨干网络中嵌入空间位置信息融合注意力机制(Spatial Location Information Fusion,SLIF)弥补小目标特征信息。其次,结合C3模块和动态蛇形卷积提出多视角特征提取(Multi-view Feature Extraction,MVFE)模块,通过在不同视角下提取同一特征来增强小目标的特征表达能力。采用大选择核(Large Selection Kernel,LSK)模块,通过使用不同大小的卷积核提取小目标多尺度信息,以提高对红外小目标定位能力。最后,引入基于注意力的尺度内特征交互(Attention-based Intrascale Feature Interaction,AIFI)模块增强特征之间的交互性。在对空红外小目标数据集上进行实验,实验结果表明,mAP75的检测精度为90.5%,mAP50~95检测精度为74.5%,文中模型能够较好地实现对红外小目标精确检测。展开更多
点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factoriza...点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factorization machine,Self-AtDFEFM)模型。首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测。在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和LogLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验。实验结果表明:在两大数据集上,Self-AtDFEFM模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大。Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性。展开更多
基金supported by the National Key Research and Development Project(No.2020YFC1512000)the General Projects of Key R&D Programs in Shaanxi Province(No.2020GY-060)Xi’an Science&Technology Project(No.2020KJRC 0126)。
文摘With the development of sensors,the application of multi-source remote sensing data has been widely concerned.Since hyperspectral image(HSI)contains rich spectral information while light detection and ranging(LiDAR)data contains elevation information,joint use of them for ground object classification can yield positive results,especially by building deep networks.Fortu-nately,multi-scale deep networks allow to expand the receptive fields of convolution without causing the computational and training problems associated with simply adding more network layers.In this work,a multi-scale feature fusion network is proposed for the joint classification of HSI and LiDAR data.First,we design a multi-scale spatial feature extraction module with cross-channel connections,by which spatial information of HSI data and elevation information of LiDAR data are extracted and fused.In addition,a multi-scale spectral feature extraction module is employed to extract the multi-scale spectral features of HSI data.Finally,joint multi-scale features are obtained by weighting and concatenation operations and then fed into the classifier.To verify the effective-ness of the proposed network,experiments are carried out on the MUUFL Gulfport and Trento datasets.The experimental results demonstrate that the classification performance of the proposed method is superior to that of other state-of-the-art methods.
文摘In this study,an underwater image enhancement method based on multi-scale adversarial network was proposed to solve the problem of detail blur and color distortion in underwater images.Firstly,the local features of each layer were enhanced into the global features by the proposed residual dense block,which ensured that the generated images retain more details.Secondly,a multi-scale structure was adopted to extract multi-scale semantic features of the original images.Finally,the features obtained from the dual channels were fused by an adaptive fusion module to further optimize the features.The discriminant network adopted the structure of the Markov discriminator.In addition,by constructing mean square error,structural similarity,and perceived color loss function,the generated image is consistent with the reference image in structure,color,and content.The experimental results showed that the enhanced underwater image deblurring effect of the proposed algorithm was good and the problem of underwater image color bias was effectively improved.In both subjective and objective evaluation indexes,the experimental results of the proposed algorithm are better than those of the comparison algorithm.
文摘Offshore carbon dioxide(CO_(2)) geological storage(OCGS) represents a significant strategy for addressing climate change by curtailing greenhouse gas emissions. Nonetheless, the risk of CO_(2) leakage poses a substantial concern associated with this technology. This study introduces an innovative approach for establishing OCGS leakage scenarios, involving four pivotal stages, namely, interactive matrix establishment, risk matrix evaluation, cause–effect analysis, and scenario development, which has been implemented in the Pearl River Estuary Basin in China. The initial phase encompassed the establishment of an interaction matrix for OCGS systems based on features, events, and processes. Subsequent risk matrix evaluation and cause–effect analysis identified key system components, specifically CO_(2) injection and faults/features. Building upon this analysis, two leakage risk scenarios were successfully developed, accompanied by the corresponding mitigation measures. In addition, this study introduces the application of scenario development to risk assessment, including scenario numerical simulation and quantitative assessment. Overall, this research positively contributes to the sustainable development and safe operation of OCGS projects and holds potential for further refinement and broader application to diverse geographical environments and project requirements. This comprehensive study provides valuable insights into the establishment of OCGS leakage scenarios and demonstrates their practical application to risk assessment, laying the foundation for promoting the sustainable development and safe operation of ocean CO_(2) geological storage projects while proposing possibilities for future improvements and broader applications to different contexts.
基金supported by the International Research Center of Big Data for Sustainable Development Goals under Grant No.CBAS2022GSP05the Open Fund of State Key Laboratory of Remote Sensing Science under Grant No.6142A01210404the Hubei Key Laboratory of Intelligent Geo-Information Processing under Grant No.KLIGIP-2022-B03.
文摘Named entity recognition(NER)is an important part in knowledge extraction and one of the main tasks in constructing knowledge graphs.In today’s Chinese named entity recognition(CNER)task,the BERT-BiLSTM-CRF model is widely used and often yields notable results.However,recognizing each entity with high accuracy remains challenging.Many entities do not appear as single words but as part of complex phrases,making it difficult to achieve accurate recognition using word embedding information alone because the intricate lexical structure often impacts the performance.To address this issue,we propose an improved Bidirectional Encoder Representations from Transformers(BERT)character word conditional random field(CRF)(BCWC)model.It incorporates a pre-trained word embedding model using the skip-gram with negative sampling(SGNS)method,alongside traditional BERT embeddings.By comparing datasets with different word segmentation tools,we obtain enhanced word embedding features for segmented data.These features are then processed using the multi-scale convolution and iterated dilated convolutional neural networks(IDCNNs)with varying expansion rates to capture features at multiple scales and extract diverse contextual information.Additionally,a multi-attention mechanism is employed to fuse word and character embeddings.Finally,CRFs are applied to learn sequence constraints and optimize entity label annotations.A series of experiments are conducted on three public datasets,demonstrating that the proposed method outperforms the recent advanced baselines.BCWC is capable to address the challenge of recognizing complex entities by combining character-level and word-level embedding information,thereby improving the accuracy of CNER.Such a model is potential to the applications of more precise knowledge extraction such as knowledge graph construction and information retrieval,particularly in domain-specific natural language processing tasks that require high entity recognition precision.
文摘In this paper,based on a bidirectional parallel multi-branch feature pyramid network(BPMFPN),a novel one-stage object detector called BPMFPN Det is proposed for real-time detection of ground multi-scale targets by swarm unmanned aerial vehicles(UAVs).First,the bidirectional parallel multi-branch convolution modules are used to construct the feature pyramid to enhance the feature expression abilities of different scale feature layers.Next,the feature pyramid is integrated into the single-stage object detection framework to ensure real-time performance.In order to validate the effectiveness of the proposed algorithm,experiments are conducted on four datasets.For the PASCAL VOC dataset,the proposed algorithm achieves the mean average precision(mAP)of 85.4 on the VOC 2007 test set.With regard to the detection in optical remote sensing(DIOR)dataset,the proposed algorithm achieves 73.9 mAP.For vehicle detection in aerial imagery(VEDAI)dataset,the detection accuracy of small land vehicle(slv)targets reaches 97.4 mAP.For unmanned aerial vehicle detection and tracking(UAVDT)dataset,the proposed BPMFPN Det achieves the mAP of 48.75.Compared with the previous state-of-the-art methods,the results obtained by the proposed algorithm are more competitive.The experimental results demonstrate that the proposed algorithm can effectively solve the problem of real-time detection of ground multi-scale targets in aerial images of swarm UAVs.
基金the National Natural Science Foundation of China(32172240)BL19U2 beamline of National Facility for Protein Science in Shanghai(NFPS)at Shanghai Synchrotron Radiation Facility,for their assistance during data collection。
文摘Three materials(agar,konjac glucomannan(KGM)andκ-carrageenan)were used to prepare ternary systems,i.e.,sol-gels and their dried composites conditioned at varied relative humidity(RH)(33%,54%and 75%).Combined methods,e.g.,scanning electron microscopy,small-angle X-ray scattering,infrared spectroscopy(IR)and X-ray diffraction(XRD),were used to disclose howκ-carrageenan addition tailors the features of agar/KGM/κ-carrageenan ternary system.As affirmed by IR and XRD,the ternary systems withκ-carrageenan below 25%(agar/KGM/carrageenan,50:25:25,m/m)displayed proper component interactions,which increased the sol-gel transition temperature and the hardness of obtained gels.For instance,the ternary composites could show hardness about 3 to 4 times higher than that for binary counterpart.These gels were dehydrated to acquire ternary composites.Compared to agar/KGM composite,the ternary composites showed fewer crystallites and nanoscale orders,and newly-formed nanoscale structures from chain assembly.Such multi-scale structures,for composites withκ-carrageenan below 25%,showed weaker changes with RH,as revealed by especially morphologic and crystalline features.Consequently,the ternary composites with lessκ-carrageenan(below 25%)exhibited stabilized elongation at break and hydrophilicity at different RHs.This hints to us that agar/KGM/κ-carrageenan composite systems can display series applications with improved features,e.g.,increased sol-gel transition point.
文摘六自由度(Six Degrees of Freedom,6DoF)视频允许用户从全方位、任意视角身临其境体验场景,是下一代沉浸式视频产业的发展方向.部分自由度受限的窗口6DoF视频近年来成为研究热点,本文提出面向窗口6DoF合成视频的主观数据库和客观质量评价方法.在主观数据库方面,构建了包含两种交互路径不适性失真、四种绘制失真和四种压缩失真的窗口6DoF合成视频主观质量数据库Windowed-6DoF,并开展主观质量测试及结果分析.在客观质量评价方法方面,设计了一种融合多层特征的窗口6DoF合成视频无参考客观质量评价方法.采用切比雪夫矩提取视频时域切片上的底层形状特征;采用Resnet-50网络提取视频的时域、空域高层语义特征并进行降维处理;最后采用随机森林将底层形状特征和高层语义特征进行融合,且训练得到窗口6DoF合成视频的客观质量评价模型.在提出的数据库Windowed-6DoF和公共数据库IRCCyN/IVC DIBR的测试结果表明,本文提出的客观质量评价方法预测分数的皮尔逊线性相关系数分别达到0.9327和0.8581,与主观评价分数具有较好的一致性.
基金support from the Ministry of Education(MOE) Singapore Tier 1 (RG8/20)。
文摘A large database is desired for machine learning(ML) technology to make accurate predictions of materials physicochemical properties based on their molecular structure.When a large database is not available,the development of proper featurization method based on physicochemical nature of target proprieties can improve the predictive power of ML models with a smaller database.In this work,we show that two new featurization methods,volume occupation spatial matrix and heat contribution spatial matrix,can improve the accuracy in predicting energetic materials' crystal density(ρ_(crystal)) and solid phase enthalpy of formation(H_(f,solid)) using a database containing 451 energetic molecules.Their mean absolute errors are reduced from 0.048 g/cm~3 and 24.67 kcal/mol to 0.035 g/cm~3 and 9.66 kcal/mol,respectively.By leave-one-out-cross-validation,the newly developed ML models can be used to determine the performance of most kinds of energetic materials except cubanes.Our ML models are applied to predict ρ_(crystal) and H_(f,solid) of CHON-based molecules of the 150 million sized PubChem database,and screened out 56 candidates with competitive detonation performance and reasonable chemical structures.With further improvement in future,spatial matrices have the potential of becoming multifunctional ML simulation tools that could provide even better predictions in wider fields of materials science.
文摘点击率(CTR)预测通过预测用户对广告或商品的点击概率,实现数字广告精准推荐。针对现有CTR模型存在原始嵌入向量未精化、特征交互方式偏简单的问题,本文提出自注意力深度域嵌入因子分解机(self-attention deep field-embedded factorization machine,Self-AtDFEFM)模型。首先,通过多头自注意力对原始嵌入向量加权,精化出关键低层特征;其次,构建深度域嵌入因子分解机(FEFM)模块,设计域对对称矩阵以提升不同特征域之间的交互强度,为高阶特征交互优选出低阶特征组合;再次,基于低阶特征组合构建深度神经网络(DNN),完成隐式高阶特征交互;然后,围绕精化后的嵌入向量,联合多头自注意力与残差机制堆叠多个显式高阶特征交互层,通过自注意力捕获同一特征在不同子空间上的互补信息,完成显示高阶特征交互;最后,联合显式与隐式高阶特征交互实现点击率预测。在Criteo和Avazu两大公开数据集上,将Self-AtDFEFM模型与主流基线模型在AUC和LogLoss指标上进行对比实验;为Self-AtDFEFM模型调制显式高阶特征交互层层数、注意力头数量、嵌入层维度及隐式高阶特征交互层层数等参数;对Self-AtDFEFM模型进行消融实验。实验结果表明:在两大数据集上,Self-AtDFEFM模型的AUC、LogLoss均优于主流基线模型;Self-AtDFEFM模型的全部参数已调为最佳;各模块形成合力以促使Self-AtDFEFM模型性能达到最优,其中显示高阶特征交互层的作用最大。Self-AtDFEFM模型各模块即插即用,易于构建和部署,且在性能与复杂度之间取得平衡,具备较高实用性。
