Improved real-time data anomaly detection using context classification
Апстракт
The number of automated measuring and reporting systems used in water distribution and sewer systems is dramatically increasing and, as a consequence, so is the volume of data acquired. Since real-time data is likely to contain a certain amount of anomalous values and data acquisition equipment is not perfect, it is essential to equip the SCADA (Supervisory Control and Data Acquisition) system with automatic procedures that can detect the related problems and assist the user in monitoring and managing the incoming data. A number of different anomaly detection techniques and methods exist and can be used with varying success. To improve the performance, these methods must be fine tuned according to crucial aspects of the process monitored and the contexts in which the data are classified. The aim of this paper is to explore if the data context classification and pre-processing techniques can be used to improve the anomaly detection methods, especially in fully automated systems. The met...hodology developed is tested on sets of real-life data, using different standard and experimental anomaly detection procedures including statistical, model-based and data-mining approaches. The results obtained clearly demonstrate the effectiveness of the suggested anomaly detection methodology.
Кључне речи:
anomaly detection / context-classification-based detection / data pre-processing / sewer dataИзвор:
Journal of Hydroinformatics, 2011, 13, 3, 307-323Издавач:
- IWA Publishing
DOI: 10.2166/hydro.2011.042
ISSN: 1464-7141
WoS: 000292538300002
Scopus: 2-s2.0-79959797644
Колекције
Институција/група
GraFarTY - JOUR AU - Branisavljević, Nemanja AU - Kapelan, Zoran AU - Prodanović, Dušan PY - 2011 UR - https://grafar.grf.bg.ac.rs/handle/123456789/363 AB - The number of automated measuring and reporting systems used in water distribution and sewer systems is dramatically increasing and, as a consequence, so is the volume of data acquired. Since real-time data is likely to contain a certain amount of anomalous values and data acquisition equipment is not perfect, it is essential to equip the SCADA (Supervisory Control and Data Acquisition) system with automatic procedures that can detect the related problems and assist the user in monitoring and managing the incoming data. A number of different anomaly detection techniques and methods exist and can be used with varying success. To improve the performance, these methods must be fine tuned according to crucial aspects of the process monitored and the contexts in which the data are classified. The aim of this paper is to explore if the data context classification and pre-processing techniques can be used to improve the anomaly detection methods, especially in fully automated systems. The methodology developed is tested on sets of real-life data, using different standard and experimental anomaly detection procedures including statistical, model-based and data-mining approaches. The results obtained clearly demonstrate the effectiveness of the suggested anomaly detection methodology. PB - IWA Publishing T2 - Journal of Hydroinformatics T1 - Improved real-time data anomaly detection using context classification EP - 323 IS - 3 SP - 307 VL - 13 DO - 10.2166/hydro.2011.042 ER -
@article{ author = "Branisavljević, Nemanja and Kapelan, Zoran and Prodanović, Dušan", year = "2011", abstract = "The number of automated measuring and reporting systems used in water distribution and sewer systems is dramatically increasing and, as a consequence, so is the volume of data acquired. Since real-time data is likely to contain a certain amount of anomalous values and data acquisition equipment is not perfect, it is essential to equip the SCADA (Supervisory Control and Data Acquisition) system with automatic procedures that can detect the related problems and assist the user in monitoring and managing the incoming data. A number of different anomaly detection techniques and methods exist and can be used with varying success. To improve the performance, these methods must be fine tuned according to crucial aspects of the process monitored and the contexts in which the data are classified. The aim of this paper is to explore if the data context classification and pre-processing techniques can be used to improve the anomaly detection methods, especially in fully automated systems. The methodology developed is tested on sets of real-life data, using different standard and experimental anomaly detection procedures including statistical, model-based and data-mining approaches. The results obtained clearly demonstrate the effectiveness of the suggested anomaly detection methodology.", publisher = "IWA Publishing", journal = "Journal of Hydroinformatics", title = "Improved real-time data anomaly detection using context classification", pages = "323-307", number = "3", volume = "13", doi = "10.2166/hydro.2011.042" }
Branisavljević, N., Kapelan, Z.,& Prodanović, D.. (2011). Improved real-time data anomaly detection using context classification. in Journal of Hydroinformatics IWA Publishing., 13(3), 307-323. https://doi.org/10.2166/hydro.2011.042
Branisavljević N, Kapelan Z, Prodanović D. Improved real-time data anomaly detection using context classification. in Journal of Hydroinformatics. 2011;13(3):307-323. doi:10.2166/hydro.2011.042 .
Branisavljević, Nemanja, Kapelan, Zoran, Prodanović, Dušan, "Improved real-time data anomaly detection using context classification" in Journal of Hydroinformatics, 13, no. 3 (2011):307-323, https://doi.org/10.2166/hydro.2011.042 . .