在传感器网络中,有监督的异常数据检测算法的检测准确率以及鲁棒性受限于有标注数据集的构建,无监督异常数据检测算法往往导致较高的误报率(FPR)。为解决上述问题,针对到达服务器端的传感器数据流提出了一种基于卷积神经网络(CNN)和长...在传感器网络中,有监督的异常数据检测算法的检测准确率以及鲁棒性受限于有标注数据集的构建,无监督异常数据检测算法往往导致较高的误报率(FPR)。为解决上述问题,针对到达服务器端的传感器数据流提出了一种基于卷积神经网络(CNN)和长短时记忆网络(LSTM)的半监督在线异常检测算法。本算法利用K-means判别检测误差,并在检测中利用新数据重新训练机器学习模型,从而提高模型在长时间范围内的异常检测准确度。为了评估本算法的性能,使用因特尔伯克利实验室数据集IBRL(Intel Berkeley Research Lab)完成仿真实验,并与同类算法进行对比。实验结果表明,与同类算法相比,本算法对各个数据集都具有较高的召回率和F1-Score;应用K-means聚类的半监督模型,其异常检测结果更稳定。展开更多
The detection of outliers and change points from time series has become research focus in the area of time series data mining since it can be used for fraud detection, rare event discovery, event/trend change detectio...The detection of outliers and change points from time series has become research focus in the area of time series data mining since it can be used for fraud detection, rare event discovery, event/trend change detection, etc. In most previous works, outlier detection and change point detection have not been related explicitly and the change point detections did not consider the influence of outliers, in this work, a unified detection framework was presented to deal with both of them. The framework is based on ALARCON-AQUINO and BARRIA's change points detection method and adopts two-stage detection to divide the outliers and change points. The advantages of it lie in that: firstly, unified structure for change detection and outlier detection further reduces the computational complexity and make the detective procedure simple; Secondly, the detection strategy of outlier detection before change point detection avoids the influence of outliers to the change point detection, and thus improves the accuracy of the change point detection. The simulation experiments of the proposed method for both model data and actual application data have been made and gotten 100% detection accuracy. The comparisons between traditional detection method and the proposed method further demonstrate that the unified detection structure is more accurate when the time series are contaminated by outliers.展开更多
文摘在传感器网络中,有监督的异常数据检测算法的检测准确率以及鲁棒性受限于有标注数据集的构建,无监督异常数据检测算法往往导致较高的误报率(FPR)。为解决上述问题,针对到达服务器端的传感器数据流提出了一种基于卷积神经网络(CNN)和长短时记忆网络(LSTM)的半监督在线异常检测算法。本算法利用K-means判别检测误差,并在检测中利用新数据重新训练机器学习模型,从而提高模型在长时间范围内的异常检测准确度。为了评估本算法的性能,使用因特尔伯克利实验室数据集IBRL(Intel Berkeley Research Lab)完成仿真实验,并与同类算法进行对比。实验结果表明,与同类算法相比,本算法对各个数据集都具有较高的召回率和F1-Score;应用K-means聚类的半监督模型,其异常检测结果更稳定。
基金Project(2011AA040603) supported by the National High Technology Ressarch & Development Program of ChinaProject(201202226) supported by the Natural Science Foundation of Liaoning Province, China
文摘The detection of outliers and change points from time series has become research focus in the area of time series data mining since it can be used for fraud detection, rare event discovery, event/trend change detection, etc. In most previous works, outlier detection and change point detection have not been related explicitly and the change point detections did not consider the influence of outliers, in this work, a unified detection framework was presented to deal with both of them. The framework is based on ALARCON-AQUINO and BARRIA's change points detection method and adopts two-stage detection to divide the outliers and change points. The advantages of it lie in that: firstly, unified structure for change detection and outlier detection further reduces the computational complexity and make the detective procedure simple; Secondly, the detection strategy of outlier detection before change point detection avoids the influence of outliers to the change point detection, and thus improves the accuracy of the change point detection. The simulation experiments of the proposed method for both model data and actual application data have been made and gotten 100% detection accuracy. The comparisons between traditional detection method and the proposed method further demonstrate that the unified detection structure is more accurate when the time series are contaminated by outliers.
