Many classical clustering algorithms do good jobs on their prerequisite but do not scale well when being applied to deal with very large data sets(VLDS).In this work,a novel division and partition clustering method(DP...Many classical clustering algorithms do good jobs on their prerequisite but do not scale well when being applied to deal with very large data sets(VLDS).In this work,a novel division and partition clustering method(DP) was proposed to solve the problem.DP cut the source data set into data blocks,and extracted the eigenvector for each data block to form the local feature set.The local feature set was used in the second round of the characteristics polymerization process for the source data to find the global eigenvector.Ultimately according to the global eigenvector,the data set was assigned by criterion of minimum distance.The experimental results show that it is more robust than the conventional clusterings.Characteristics of not sensitive to data dimensions,distribution and number of nature clustering make it have a wide range of applications in clustering VLDS.展开更多
随着互联电网运行方式的愈加复杂多变以及广域量测系统部署的越来越完善,以广域测量系统(wide area measurement system,WAMS)量测大数据为基础的实时稳定分析成为必然要求。与此同时,如何对全网多节点毫秒级海量WAMS大数据进行时空同...随着互联电网运行方式的愈加复杂多变以及广域量测系统部署的越来越完善,以广域测量系统(wide area measurement system,WAMS)量测大数据为基础的实时稳定分析成为必然要求。与此同时,如何对全网多节点毫秒级海量WAMS大数据进行时空同步处理和异常数据检测,成为阻碍其发挥更大作用的关键问题。因此,该文提出基于高维随机矩阵描述的WAMS量测大数据建模与分析方法。首先在对WAMS量测数据时空特性分析的基础上,根据高维随机矩阵理论,进行了WAMS量测大数据的高维随机矩阵模型构建,然后推导了其异常数据检测理论和方法,最后在仿真算例上模拟实测量测数据,通过对比不同异常时刻量测数据的Trace检测和谱分布,验证了该量测大数据的建模方法的有效性与适用性。展开更多
基金Supported by National Natural Science Foundation of China(60675039)National High Technology Research and Development Program of China(863 Program)(2006AA04Z217)Hundred Talents Program of Chinese Academy of Sciences
基金Projects(60903082,60975042)supported by the National Natural Science Foundation of ChinaProject(20070217043)supported by the Research Fund for the Doctoral Program of Higher Education of China
文摘Many classical clustering algorithms do good jobs on their prerequisite but do not scale well when being applied to deal with very large data sets(VLDS).In this work,a novel division and partition clustering method(DP) was proposed to solve the problem.DP cut the source data set into data blocks,and extracted the eigenvector for each data block to form the local feature set.The local feature set was used in the second round of the characteristics polymerization process for the source data to find the global eigenvector.Ultimately according to the global eigenvector,the data set was assigned by criterion of minimum distance.The experimental results show that it is more robust than the conventional clusterings.Characteristics of not sensitive to data dimensions,distribution and number of nature clustering make it have a wide range of applications in clustering VLDS.
文摘随着互联电网运行方式的愈加复杂多变以及广域量测系统部署的越来越完善,以广域测量系统(wide area measurement system,WAMS)量测大数据为基础的实时稳定分析成为必然要求。与此同时,如何对全网多节点毫秒级海量WAMS大数据进行时空同步处理和异常数据检测,成为阻碍其发挥更大作用的关键问题。因此,该文提出基于高维随机矩阵描述的WAMS量测大数据建模与分析方法。首先在对WAMS量测数据时空特性分析的基础上,根据高维随机矩阵理论,进行了WAMS量测大数据的高维随机矩阵模型构建,然后推导了其异常数据检测理论和方法,最后在仿真算例上模拟实测量测数据,通过对比不同异常时刻量测数据的Trace检测和谱分布,验证了该量测大数据的建模方法的有效性与适用性。
