摘要
多源异构数据给复杂信息系统的智能化信息处理带来巨大挑战,如何对多源异构信息大数据进行融合已成为当前的热点研究技术。该文以复杂电力信息系统为基础,分析多源异构数据融合的预处理方法及其相关性分析技术;随后探讨当前主流的多源异构数据融合技术及其优缺点;在此基础上,提出基于Kafka和Storm数据处理平台的海量日志处理实际工程大数据分析架构,并利用该架构对多源异构数据融合技术进行测试和分析。该文的理论分析和实验构建可以为相关大数据工程分析与应用提供借鉴意义。
Multi-source heterogeneous data brings great challenges to the intelligent signal processing of complex information systems,and currently the fusion processing technology of multi-source heterogeneous information big data has become a research hotspot.Based on the complex power information system,the preprocessing methods of multi-source heterogeneous data fusion and its correlation analysis technology were analyzed in this paper.The current multi-source heterogeneous data fusion methods were discussed in detail,and with the comparison of the advantages and disadvantages of each technology.On this basis,a big data analysis framework for actual log processing case based on Kafka and Storm data processing platforms was proposed,and the proposed architecture is used to test and analyze multi-source heterogeneous data fusion technologies.The theoretical analysis and experimental results in this paper can provide reference for big data engineering analysis and application.
作者
林瑀
陈日成
金涛
LIN Yu;CHEN Richeng;JIN Tao(Fujian Normal University,Fuzhou 350117,China;College of Electrical Engineering and Automation,Fuzhou University,Fuzhou 350116,China)
出处
《中国测试》
CAS
北大核心
2020年第7期1-7,23,共8页
China Measurement & Test
基金
国家自然科学基金(51977039)
福建省自然科学基金(2018J05118)。
关键词
复杂信息系统
多源异构
数据融合
大数据分析
complex information system
multi-source heterogeneous
data fusion
big data analysis
作者简介
林瑀(1980-),女,福建福州市人,讲师,博士,研究方向为大数据、人工智能及其分析应用;通信作者:金涛(1976-),男,河北秦皇岛市人,教授,博士生导师,研究方向为电力系统人工智能、在线测量与信号处理等。
