Traditional named entity recognition methods need professional domain knowl-edge and a large amount of human participation to extract features,as well as the Chinese named entity recognition method based on a neural n...Traditional named entity recognition methods need professional domain knowl-edge and a large amount of human participation to extract features,as well as the Chinese named entity recognition method based on a neural network model,which brings the prob-lem that vector representation is too singular in the process of character vector representa-tion.To solve the above problem,we propose a Chinese named entity recognition method based on the BERT-BiLSTM-ATT-CRF model.Firstly,we use the bidirectional encoder representations from transformers(BERT)pre-training language model to obtain the se-mantic vector of the word according to the context information of the word;Secondly,the word vectors trained by BERT are input into the bidirectional long-term and short-term memory network embedded with attention mechanism(BiLSTM-ATT)to capture the most important semantic information in the sentence;Finally,the conditional random field(CRF)is used to learn the dependence between adjacent tags to obtain the global optimal sentence level tag sequence.The experimental results show that the proposed model achieves state-of-the-art performance on both Microsoft Research Asia(MSRA)corpus and people’s daily corpus,with F1 values of 94.77% and 95.97% respectively.展开更多
为了提高电力集控系统安全隐患数据处理的效果,提出一种基于来自变换器的双向编码器表示-双向长短期记忆网络-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short Term Memory-Conditional ...为了提高电力集控系统安全隐患数据处理的效果,提出一种基于来自变换器的双向编码器表示-双向长短期记忆网络-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short Term Memory-Conditional Random Fields,BERT-BiLSTM-CRF)的电力集控安全隐患数据处理方法。构建电力集控隐患数据检测模型,应用改进长短时记忆网络(Long Short Term Memory,LSTM)来构建电力集控安全隐患数据修复网络,实现电力集控安全隐患数据处理。实验结果表明,采用所提方法能够更好地完成电力集控安全隐患数据检测与修复,应用效果较好。展开更多
针对目前最先进的临床命名实体识别(Cinical Named Entity Recognition,CNER)模型未能充分挖掘文本的全局信息和语义特征,以及未能解决文本中的字符替换等问题,改进了传统的单词嵌入模型,并在此基础上提出了一种结合深度卷积神经网络和...针对目前最先进的临床命名实体识别(Cinical Named Entity Recognition,CNER)模型未能充分挖掘文本的全局信息和语义特征,以及未能解决文本中的字符替换等问题,改进了传统的单词嵌入模型,并在此基础上提出了一种结合深度卷积神经网络和双向短时记忆条件随机场(DCNN-BiLSTM-CRF)的临床文本命名实体识别方法。改进的单词嵌入模型融合词根、拼音和字符本身意义,使用了来自Transformers的双向编码器表示,使单词嵌入向量具有汉字和临床文本的特点,该方法通过在临床命名实体识别任务中引入深度卷积神经网络(Deep Convolutional Neural Networks,DCNN),解决了CNN预测时丢失部分信息无法找回的问题。通过使用DCNN,本文模型能够更有效地捕获全局信息、获取字符之间的权重关系和多层次语义特征信息,从而提高了临床命名实体识别的准确性。在数据集CCKS2017和CCKS2018上分别进行实验,实验结果表明,与基准模型相比,该模型F1值分别改善了0.48%,0.68%,0.6%,0.58%,0.04%和1.43%,2.36%,3.31%,1.11%,0.17%。为了进一步验证本文的模型,进行了两种消融实验。结果表明,在两个数据集CCKS2017和CCKS2018上本文模型对比变体模型M1,F1值分别改善了0.79%和0.84%;对比变体模型M2,F1值分别改善了0.53%和0.64%。这些实验结果证明了本文所提算法的可行性。展开更多
文摘Traditional named entity recognition methods need professional domain knowl-edge and a large amount of human participation to extract features,as well as the Chinese named entity recognition method based on a neural network model,which brings the prob-lem that vector representation is too singular in the process of character vector representa-tion.To solve the above problem,we propose a Chinese named entity recognition method based on the BERT-BiLSTM-ATT-CRF model.Firstly,we use the bidirectional encoder representations from transformers(BERT)pre-training language model to obtain the se-mantic vector of the word according to the context information of the word;Secondly,the word vectors trained by BERT are input into the bidirectional long-term and short-term memory network embedded with attention mechanism(BiLSTM-ATT)to capture the most important semantic information in the sentence;Finally,the conditional random field(CRF)is used to learn the dependence between adjacent tags to obtain the global optimal sentence level tag sequence.The experimental results show that the proposed model achieves state-of-the-art performance on both Microsoft Research Asia(MSRA)corpus and people’s daily corpus,with F1 values of 94.77% and 95.97% respectively.
文摘为了提高电力集控系统安全隐患数据处理的效果,提出一种基于来自变换器的双向编码器表示-双向长短期记忆网络-条件随机场(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short Term Memory-Conditional Random Fields,BERT-BiLSTM-CRF)的电力集控安全隐患数据处理方法。构建电力集控隐患数据检测模型,应用改进长短时记忆网络(Long Short Term Memory,LSTM)来构建电力集控安全隐患数据修复网络,实现电力集控安全隐患数据处理。实验结果表明,采用所提方法能够更好地完成电力集控安全隐患数据检测与修复,应用效果较好。
文摘针对现有的中文命名实体识别算法没有充分考虑实体识别任务的数据特征,存在中文样本数据的类别不平衡、训练数据中的噪声太大和每次模型生成数据的分布差异较大的问题,提出了一种以BERT-BiLSTM-CRF(Bidirectional Encoder Representations from Transformers-Bidirectional Long Short-Term Memory-Conditional Random Field)为基线改进的中文命名实体识别模型。首先在BERT-BiLSTM-CRF模型上结合P-Tuning v2技术,精确提取数据特征,然后使用3个损失函数包括聚焦损失(Focal Loss)、标签平滑(Label Smoothing)和KL Loss(Kullback-Leibler divergence loss)作为正则项参与损失计算。实验结果表明,改进的模型在Weibo、Resume和MSRA(Microsoft Research Asia)数据集上的F 1得分分别为71.13%、96.31%、95.90%,验证了所提算法具有更好的性能,并且在不同的下游任务中,所提算法易于与其他的神经网络结合与扩展。
