摘要
通过分析网络数据采集的特点,提出了网络数据采集系统的设计目标,即支持关键网络指标实时计算和查询、支持多数据源和多消费者、支持实时采集和批量采集且具备线性扩展能力。采用Flume、Kafka、Storm、Hadoop等开源技术完成了系统架构设计。对架构实施可能面临的挑战提出了应对策略。
Based on analyzing the characteristics of network data collection, it presents the design goal of network data collecting sys- tem, that is supporting the real-time computing and querying of the key network index, supporting multi-data source and multi-consumer, supporting real-time collection and batch collection, and also has the linear extend capability. The system ar- chitecture is designed with the open source technologies, such as Flume, Kafka, Storm and Hadoop. The countermeasures are presented for the challenge before architecture deployment.
出处
《邮电设计技术》
2015年第12期29-32,共4页
Designing Techniques of Posts and Telecommunications
作者简介
尧炜,毕业于北京邮电大学，工程师．硕士。主要从事信息化相关咨询，设计以及软件开发工作。
