李佳宣, 郑晓薇, 张永. 基于Poisson过程的Hadoop集群平均寿命计算方法[J]. 微电子学与计算机, 2016, 33(7): 150-153, 158.
引用本文: 李佳宣, 郑晓薇, 张永. 基于Poisson过程的Hadoop集群平均寿命计算方法[J]. 微电子学与计算机, 2016, 33(7): 150-153, 158.
LI Jia-xuan, ZHENG Xiao-wei, ZHANG Yong. A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process[J]. Microelectronics & Computer, 2016, 33(7): 150-153, 158.
Citation: LI Jia-xuan, ZHENG Xiao-wei, ZHANG Yong. A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process[J]. Microelectronics & Computer, 2016, 33(7): 150-153, 158.

基于Poisson过程的Hadoop集群平均寿命计算方法

A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process

  • 摘要: Hadoop集群在连续长时间执行任务的过程中, 集群故障会对集群造成不同程度的冲击与损害, 是集群性能下降、系统故障频发的主要原因, 更会导致集群平均寿命的大大缩短.为解决此问题, 提出了一种基于Poisson过程的Hadoop集群平均寿命预测方法.通过预测计算, 及时对集群进行调整优化, 使系统性能保持在正常状态.同时在三种不同特征类型的Hadoop集群的实验中观察集群在任务执行过程中平均寿命的变化.实验结果表明, 该方法能有效地判断Hadoop集群故障发生的频率, 预测出集群的平均寿命, 对集群性能优劣进行评价并及时对集群性能进行优化

     

    Abstract: After a long task execution on Hadoop cluster, as cluster faults would hurt Hadoop cluster on different strength, which became the main reason of cluster performance degradation and more frequent cluster faults, more serious would cause the decrease of cluster average life. To solve the problem, this paper proposes a method of Hadoop cluster average life prediction based on Poisson process. By prediction calculation, adjust and optimize the cluster timely to keep the cluster performance in a normal state. Then make several experiments on three different feature types of Hadoop cluster to observe the frequncy of cluster average life. According to the experiment results, this method is the significant to Hadoop cluster performance evaluation and optimization.

     

/

返回文章
返回