To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the sch...To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.展开更多
The increasing network throughput challenges the current network traffic monitor systems to have compatible high-performance data processing.The design of packet processing systems is guided by the requirements of hig...The increasing network throughput challenges the current network traffic monitor systems to have compatible high-performance data processing.The design of packet processing systems is guided by the requirements of high packet processing throughput.In this paper,we depict an in-depth research on the related techniques and an implementation of a high-performance data acquisition mechanism.Through the bottleneck analysis with the aid of queuing network model,several performance optimising methods,such as service rate increasing,queue removing and model simplification,are integrated.The experiment results indicate that this approach is capable of reducing the CPU utilization ratio while improving the efficiency of data acquisition in high-speed networks.展开更多
To solve the slow congestion detection and rate convergence problems in the existing rate control based fair data collection schemes, a new fair data collection scheme is proposed, which is named the improved scheme w...To solve the slow congestion detection and rate convergence problems in the existing rate control based fair data collection schemes, a new fair data collection scheme is proposed, which is named the improved scheme with fairness or ISWF for short. In ISWF, a quick congestion detection method, which combines the queue length with traffic changes of a node, is used to solve the slow congestion detection problem, and a new solution, which adjusts the rate of sending data of a node by monitoring the channel utilization rate, is used to solve the slow convergence problem. At the same time, the probability selection method is used in ISWF to achieve the fairness of channel bandwidth utilization. Experiment and simulation results show that ISWF can effectively reduce the reaction time in detecting congestion and shorten the rate convergence process. Compared with the existing tree-based fair data collection schemes, ISWF can achieve better fairness in data collection and reduce the transmission delay effectively, and at the same time, it can increase the average network throughput by 9.1% or more.展开更多
基金supported by the National Grand Fundamental Research of China (973 Program) under Grant No. 2011CB302601the National High Technology Research and Development of China (863 Program) under GrantNo. 2013AA01A213+2 种基金the National Natural Science Foundation of China under Grant No. 60873215the Natural Science Foundation for Distinguished Young Scholars of Hunan Province under Grant No. S2010J5050Specialized Research Fund for the Doctoral Program of Higher Education under Grant No. 20124307110015
文摘To reduce the time required to complete the regeneration process of erasure codes, we propose a Tree-structured Parallel Regeneration (TPR) scheme for multiple data losses in distributed storage systems. Under the scheme, two algorithms are proposed for the construction of multiple regeneration trees, namely the edge-disjoint algorithm and edge-sharing algorithm. The edge-disjoint algorithm constructs multiple independent trees, and is simple and appropriate for environments where newcomers and their providers are distributed over a large area and have few intersections. The edge-sharing algorithm constructs multiple trees that compete to utilize the bandwidth, and make a better utilization of the bandwidth, although it needs to measure the available band-width and deal with the bandwidth changes; it is therefore difficult to implement in practical systems. The parallel regeneration for multiple data losses of TPR primarily includes two optimizations: firstly, transferring the data through the bandwidth optimized-paths in a pipe-line manner; secondly, executing data regeneration over multiple trees in parallel. To evaluate the proposal, we implement an event-based simulator and make a detailed comparison with some popular regeneration methods. The quantitative comparison results show that the use of TPR employing either the edge-disjoint algorithm or edge-sharing algorithm reduces the regeneration time significantly.
基金ACKNOWLEDGEMENT This project was supported by the National Natural Science Foundation of China under Grant No. 61170262 the National High Tech- nology Research and Development Program of China （863 Program） under Grants No. 2012AA012506, No. 2012AA012901, No. 2012- AA012903+5 种基金 the Specialised Research Fund for the Doctoral Program of Higher Education of China under Grant No. 20121103120032 the Humanity and Social Science Youth Founda- tion of Ministry of Education of China under Grant No. 13YJCZH065 the Opening Project of Key Lab of Information Network Security of Ministry of Public Security （The Third Re- search Institute of Ministry of Public Security） under Grant No. C13613 the China Postdoc- toral Science Foundation, General Program of Science and Technology Development Project of Beijing Municipal Education Commission of China under Grant No. km201410005012 the Research on Education and Teaching of Beijing University of Technology under Grant No. ER2013C24 the Beijing Municipal Natu- ral Science Foundation, Sponsored by Hunan Postdoctoral Scientific Program, Open Re- search Fund of Beijing Key Laboratory of Trusted Computing.
文摘The increasing network throughput challenges the current network traffic monitor systems to have compatible high-performance data processing.The design of packet processing systems is guided by the requirements of high packet processing throughput.In this paper,we depict an in-depth research on the related techniques and an implementation of a high-performance data acquisition mechanism.Through the bottleneck analysis with the aid of queuing network model,several performance optimising methods,such as service rate increasing,queue removing and model simplification,are integrated.The experiment results indicate that this approach is capable of reducing the CPU utilization ratio while improving the efficiency of data acquisition in high-speed networks.
基金supported in part by the National Natural Science Foundation of China under Grants No. 61103178, No. 60803151the Research Fund for the Doctoral Program of Higher Education of China under Grant No. 20096102120045
文摘To solve the slow congestion detection and rate convergence problems in the existing rate control based fair data collection schemes, a new fair data collection scheme is proposed, which is named the improved scheme with fairness or ISWF for short. In ISWF, a quick congestion detection method, which combines the queue length with traffic changes of a node, is used to solve the slow congestion detection problem, and a new solution, which adjusts the rate of sending data of a node by monitoring the channel utilization rate, is used to solve the slow convergence problem. At the same time, the probability selection method is used in ISWF to achieve the fairness of channel bandwidth utilization. Experiment and simulation results show that ISWF can effectively reduce the reaction time in detecting congestion and shorten the rate convergence process. Compared with the existing tree-based fair data collection schemes, ISWF can achieve better fairness in data collection and reduce the transmission delay effectively, and at the same time, it can increase the average network throughput by 9.1% or more.
