摘要
利用文本挖掘技术进行体育热点分析,可以为体育领域的发展提供更多有用的信息。文中提出了一种基于TF-IDF(Term Frequency-Inverse Document Frequency,词频-逆文档频率)和TextRank(文本排序)的中文文本关键词提取方法,该方法首先采用分词、去除停用词等对文本进行预处理;其次使用TF-IDF算法计算每个词的重要性并进行归一化处理,同时使用TextRank算法权衡单词之间的关系并计算每个单词的得分以进行归一化处理;最后将TF-IDF值和TextRank得分进行加权和得到每个词的综合权重值,最终获得权重值最高的N个关键词。应用TF-IDF和TextRank结合的方法在F1值上选择5个关键词时取得了更好的结果,相较于只使用TF-IDF方法或TextRank方法,其关键词提取准确率分别提高约40%和32%。该方法有效提高了关键词提取的准确性和提取效率。
Using text mining techniques for sports hot topic analysis can provide more useful information for the development of the sports field.This paper proposes a method for extracting Chinese text keywords based on TF-IDF and TextRank.This method preprocesses the text by tokenizing and removing stop words,and then calculates the importance of each word using the TF-IDF algorithm and normalizes the values.Fianlly,the TextRank algorithm is used to weigh the relationships between words and calculate scores for each word,which are also normalized.Finally,the TF-IDF values and TextRank scores are weighted to obtain a comprehensive weight for each word,ultimately obtaining the N keywords with the highest weight value.The method of combining TF-IDF and TextRank achieved better results when selecting 5 keywords on F 1 value,and compared to using only TF-IDF method or TextRank method,the accuracy of keyword extraction increases by about 40%and 32%,respectively.This method effectively improves the accuracy and efficiency of keyword extraction.
作者
兰晓芳
刘卓
许志豪
肖毅
LAN Xiaofang;LIU Zhuo;XU Zhihao;XIAO Yi(Oriental College of Science and Technology,Hunan Agricultural University,Changsha 410128,China;College of In f ormation and Intelligent,Hunan Agricultural University,Changsha 410128,China)
出处
《软件工程》
2023年第8期6-10,共5页
Software Engineering
作者简介
通信作者:兰晓芳(1998-),女,本科生。研究领域:数据处理,推荐算法,lanxf@stu.hunau.edu.cn;刘卓(2002-),男,本科生。研究领域:人工智能,数据处理,fetty_max@163.com;许志豪(2001-),男,本科生。研究领域:机器学习,数据处理,guapideyouxiang@stu.hunau.edu.cn;肖毅(1978-),男,博士生,讲师。研究领域:数据处理,模式识别, xiaoyi@hunau.edu.cn。
