摘要
Managing massive electric power data is a typical big data application because electric power systems generate millions or billions of status,debugging,and error records every single day.To guarantee the safety and sustainability of electric power systems,massive electric power data need to be processed and analyzed quickly to make real-time decisions.Traditional solutions typically use relational databases to manage electric power data.However,relational databases cannot efficiently process and analyze massive electric power data when the data size increases significantly.In this paper,we show how electric power data can be managed by using HBase,a distributed database maintained by Apache.Our system consists of clients,HBase database,status monitors,data migration modules,and data fragmentation modules.We evaluate the performance of our system through a series of experiments.We also show how HBase’s parameters can be tuned to improve the efficiency of our system.
Managing massive electric power data is a typical big data application because electric power systems generate millions or billions of status, debugging, and error records every single day. To guarantee the safety and sustainability of electric power systems, massive electric power data need to be processed and analyzed quickly to make real-time decisions. Traditional solutions typically use relational databases to manage electric power data.However, relational databases cannot efficiently process and analyze massive electric power data when the data size increases significantly. In this paper, we show how electric power data can be managed by using HBase, a distributed database maintained by Apache. Our system consists of clients, HBase database, status monitors, data migration modules, and data fragmentation modules. We evaluate the performance of our system through a series of experiments. We also show how HBase’s parameters can be tuned to improve the efficiency of our system.
基金
supported by the National Key R&D Program of China(No.2017YFB1003000)
the National Natural Science Foundation of China(Nos.61702096,61572129,61602112,61502097,61320106007,61632008,and 61702097)
the International S&T Cooperation Program of China(No.2015DFA10490)
the Natural Science Foundation of Jiangsu Province(Nos.BK20170689 and BK20160695)
the Jiangsu Provincial Key Laboratory of Network and Information Security(No.BM2003201)
the Key Laboratory of Computer Network and Information Integration of Ministry of Education of China(No.93K-9)
the SGCC Science and Technology Program“the Distributed Data Management of Physical Distribution and Logical Integration”
partially supported by the Collaborative Innovation Center of Novel Software Technology and Industrialization and Collaborative Innovation Center of Wireless Communications Technology.
作者简介
Corresponding author:Aibo Song,received the MS degree from Shandong University of Science and Technology,and the PhD degree from Southeast University,China,in 1996 and 2003 respectively.He is currently an associate professor in the School of Computer Science and Engineering,Southeast University.His current research interests include cloud computing and big data processing.E-mail:absong,220151478@seu.edu.cn;Jiahui Jin,is an assistant professor in the School of Computer Science and Engineering,Southeast University,Nanjing,China.He received the PhD degree in computer science from Southeast University in 2015.He had been a visiting PhD student at University of Massachusetts,Amherst,USA,during August 2012 to August 2014.His current research interests include large-scale data processing,distributed systems,and parallel task scheduling.E-mail:jjin,@seu.edu.cn;Huan Gong,received the BE degree in computer science and technology from Northeast University at Qinhuangdao,in 2015.He is currently a master student in the School of Computer Science and Engineering,Southeast University.His research interests include big data and online aggregation.Yingying Xue,received the BS degree in computer science and technology from Nanjing University of Science and Technology in 2015.She is currently a PhD student in the School of Computer Science and Engineering,Southeast University,China.Her research interests mainly focus on spatial retrieval and semantic search on knowledge graph.E-mail:yxue,220161558@seu.edu.cn;Mingyang Du,received the BS degree in computer science and technology from Anhui University of Finance and Economics,in 2016.He is currently a master student in the school of Computer Science and Engineering,Southeast University.His research interests include big data and cloud computing.Fang Dong,is currently an associate professor in School of Computer Science and Engineering,Southeast University,China.He received the BS and MS degrees in computer science from Nanjing University of Science&Technology,China in 2004 and 2006,respectively,and received the PhD degree in computer science from Southeast University in 2011.His current research interests include cloud computing,big data processing,and workflow scheduling.He is a member of both IEEE and ACM,and general secretary of ACM SIGCOMM China.E-mail:fdong@seu.edu.cn;Junzhou Luo,is a full professor in the School of Computer Science and Engineering,Southeast University,Nanjing,China.He received the BS degree in applied mathematics from Southeast University in 1982,and then got the MS and PhD degree in computer network both from Southeast University in 1992 and 2000,respectively.His research interests are next generation network,protocol engineering,network security and management,cloud computing,and wireless LAN.He is a member of both IEEE and ACM,and co-chair of IEEE SMC Technical Committee on Computer Supported CooperativeWork in Design.E-mail:jluo@seu.edu.cn
