Premium
An improved methodology for outlier detection in dynamic datasets
Author(s) -
Xu Shu,
Baldea Michael,
Edgar Thomas F.,
Wojsznis Willy,
Blevins Terrence,
Nixon Mark
Publication year - 2015
Publication title -
aiche journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.958
H-Index - 167
eISSN - 1547-5905
pISSN - 0001-1541
DOI - 10.1002/aic.14631
Subject(s) - outlier , anomaly detection , univariate , kalman filter , computer science , process (computing) , data mining , series (stratigraphy) , filter (signal processing) , multivariate statistics , line (geometry) , artificial intelligence , pattern recognition (psychology) , machine learning , mathematics , computer vision , paleontology , geometry , biology , operating system
A time series Kalman filter (TSKF) is proposed that successfully handles outlier detection in dynamic systems, where normal process changes often mask the existence of outliers. The TSKF method combines a time series model fitting procedure with a modified Kalman filter to deal with additive outlier and innovational outlier detection problems in dynamic process dataset. Compared with current outlier detection methods, the new method enjoys the following advantages: (a) no prior knowledge of the process model is needed; (b) it is easy to tune; (c) it can be applied to both univariate and multivariate outlier detection; (d) it is applicable to both on‐line and off‐line operation; (e) it cleans outliers while maintains the integrity of the original dataset. © 2014 American Institute of Chemical Engineers AIChE J , 61: 419–433, 2015