摘要
机器阅读理解是自然语言处理的关键任务,旨在考察机器是否具有理解自然语言的能力。目前,中英文领域的机器阅读理解取得了很大的进展,模型的性能已经超过了人类的水平,其中公开数据集起到了非常重要的作用,如斯坦福大学的SQuAD数据集。而对于低资源语言,如藏文,由于缺乏大规模公开的数据集,机器阅读理解的相关研究还处于起步阶段。因此,本文构建了面向藏文机器阅读理解的数据集,数据集包含631篇文章,903个段落,形式为excel文件。其中,第一列为当前文章ID,第二列为文本标题,第三列为段落,第四列为问题,第五列为问题对应的答案。其文本数据来自云藏网,问题和答案是由20位藏语专业人员进行人工录入和标注,类型涉及自然、文化、教育等12个领域。本数据集的发布,对于促进藏文信息处理的发展具有重要的价值。
Machine reading comprehension is a key task of natural language processing,which is aimed to check whether machines have the ability to understand natural language.At present,great progress has been made in machine reading comprehension in both Chinese and English.The performance of models has surpassed the level of human beings,in which open datasets play a very important role,such as the SQuAD dataset constructed by the Stanford University.For low-resource languages,such as Tibetan,the research on machine reading comprehension is still in its infancy due to a lack of large-scale public datasets.Therefore,in this paper we constructed a dataset for Tibetan machine reading comprehension,containing 631 articles and 903 paragraphs in the form of Excel files.The first column is the article ID;the second column is the text title;the third column is the paragraph;the fourth column is the question;and the fifth column is the answer.The data are from Yunzin website.The questions and answers are manually recorded and marked by 20 Tibetan language professionals.The data includes 12 fields,such as nature,culture and education.The publication of this dataset has great value to the development of Tibetan information processing.
作者
孙媛
旦正错
刘思思
赵小兵
SUN Yuan;DAN Zhengcuo;LIU Sisi;ZHAO Xiaobing(College of Information Engineering,Minzu University of China,Beijing 100081,P.R.China;National Language Resource Monitoring&Research Center of Minority Languages,100081,P.R.China)
基金
自然科学基金项目(No.61972436)
关键词
机器阅读理解
藏文
少数民族语言
低资源
数据集
machine reading comprehension
Tibetan
minority languages
low resources
dataset
作者简介
通信作者:孙媛(1979-),女,山东省滨州市人,博士,教授,研究方向为自然语言处理。主要承担工作:数据质量控制与综合管理、数据采集,tracy.yuan.sun@gmail.com;旦正错(1998-),女,青海省海南州人,硕士研究生,研究方向为自然语言处理。主要承担工作:数据集的预处理和整合、数据校对、论文撰写;刘思思(1998-),女,湖北随州人,硕士研究生,研究方向为自然语言处理。主要承担工作:数据采集、论文撰写;赵小兵(1967-),女,内蒙古自治区呼和浩特市人,博士,教授,研究方向为自然语言处理。主要承担工作:数据集质量控制。
