带虚警抑制的基于归一化残差的野值检测方法

汝小虎; 柳征; 姜文利; 黄知涛

doi:10.11999/JEIT150469

带虚警抑制的基于归一化残差的野值检测方法

doi: 10.11999/JEIT150469

计量
- 文章访问数: 1339
- HTML全文浏览量: 122
- PDF下载量: 424
- 被引次数: 0
出版历程
- 收稿日期: 2015-04-22
- 修回日期: 2015-09-01
- 刊出日期: 2015-12-19

Normalized Residual-based Outlier Detection with False-alarm Probability Controlling

摘要

摘要: 野值检测，或称异常值检测是模式识别和知识发现中一个重要的问题。以往的野值检测方法难以有效地抑制虚警概率，针对这一问题，该文提出一种带监督情形下基于归一化残差(Normalized Residual, NR)的野值检测方法。首先利用训练样本计算待考查模式的NR值，其次比较NR值与野值检测门限的相对大小，从而判断待考查模式是否为野值。该文理论上推导了野值门限与虚警概率之间的关系表达式，以此为依据设置检测门限，可实现在少量训练样本情况下仍能抑制虚警率的目的。计算机仿真和实测数据测试验证了所提方法在野值检测和虚警抑制方面的优越性能。
- 模式识别 /
- 监督 /
- 野值检测 /
- 虚警概率 /
- 归一化残差
Abstract: Outlier detection, also called anomaly detection, is an important issue in pattern recognition and knowledge discovery. Previous outlier detection methods can not effectively control the false-alarm probability. To solve the problem, a supervised method based on Normalized Residual (NR) is proposed. Using the training patterns, it first calculates the NR value of the query pattern, which is compared with a predefined detection threshold to determine whether the pattern is an outlier. In this paper, the relationship between the threshold and false-alarm probability is theoretically derived, based on which an appropriate threshold can be chosen. In this way, the desired false-alarm probability can be obtained even when few training patterns are available. Simulations and measured data experiments validate the superior performance of the proposed method on outlier detection and false-alarm probability controlling.
- Pattern recognition /
- Supervised /
- Outlier detection /
- False-alarm probability /
- Normalized Residual (NR)

HTML全文

参考文献(20)

Hawkins D. Identification of Outliers[M]. London: Chapman and Hall, 1980: Chapter 1-2.

Liu J, Wan J, Zheng H, et al.. A method of specific emitter verification based on CSDA and SVDD[C]. Proceedings of the IEEE 2nd International Conference on Computer Science and Network Technology, Changchun, China, 2012: 562-565.

Miller D J and Browning J. A mixture model and EM-based algorithm for class discovery, robust classification, and outlier rejection in mixed labeled/unlabeled data sets[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(11): 1468-1483.

Westerweel J and Scarano F. Universal outlier detection for PIV data[J]. Experiments in Fluids, 2005, 39(6): 1096-1100.

Duncan J, Dabiri D, Hove J, et al.. Universal outlier detection for Particle Image Velocimetry (PIV) and Particle Tracking Velocimetry (PTV) data[J]. Measurement Science and Technology, 2010, 21(5): 57002-57006.

Wu S and Wang S R. Information-theoretic outlier detection for large-scale categorical data[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(3): 589-602.

Li Z G, Baseman R J, Zhu Y D, et al.. A unified framework for outlier detection in trace data analysis[J]. IEEE Transactions on Semiconductor Manufacturing, 2014, 27(1): 95-103.

Albanese A, Pal S K, and Petrosino A. Rough sets, kernel set, and spatiotemporal outlier detection[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(1): 194-207.

Ghosh A K and Chaudhuri P. On maximum depth and related classifiers[J]. Scandinavian Journal of Statistics, 2005, 32(2): 327-350.

Ru X H, Liu Z, and Jiang W L. Normalized residual-based outlier detection[C]. Proceedings of the IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC), Guilin, China, 2014: 190-193.

Nattorn B, Arthorn L, and Krung S. Outlier detection score based on ordered distance difference[C]. Proceedings of the IEEE International Computer Science and Engineering Conference (ICSEC), Nakhon Pathom, Thailand, 2013: 157-162.

Zhao M and Saligrama V. Anomaly detection with score functions based on nearest neighbor graphs[J]. Advances in Neural Information Processing Systems, 2009, 22(1): 2250-2258.

Qian J and Saligrama V. New statistic in p-value estimation for anomaly detection[C]. Proceedings of the IEEE Statistical Signal Processing Workshop (SSP), Ann Arbor, Michigan, USA, 2012: 393-396.

Chen Y T, Qian J, and Saligrama V. A new one-class SVM for anomaly detection[C]. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, 2013: 3567-3571.

Breunig M M, Kriegel H-P, Ng R T, et al.. LOF: identifying density-based local outliers[C]. Proceedings of the ACM SIGMOD International Conference on Management of Data, New York, USA, 2000: 93-104.

Sch?lkopf B, Platt J C, Shawe-Taylor J C, et al.. Estimating the support of a high-dimensional distribution[J]. Neural Computation, 2001, 13(7): 1443-1471.

Furlani M, Tuia D, Munoz-Mari J, et al.. Discovering single classes in remote sensing images with active learning[C]. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Munich, Germany, 2012: 7341-7344.

Jumutc V and Suykens J. Multi-class supervised novelty detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(12): 2510-2523.

叶浩欢, 柳征, 姜文利. 考虑多普勒效应的脉冲无意调制特征比较[J]. 电子与信息学报, 2012, 34(11): 2654-2659.

Ye H H, Liu Z, and Jiang W L. A comparison of unintentional modulation on pulse features with the consideration of Doppler effect[J]. Journal of Electronics Information Technology, 2012, 34(11): 2654-2659.

施引文献

资源附件(0)

访问统计