Open AccessAnomaly Detection in Text Data Sets using Character-Level Representation
Author(s) -
Mahsa Mohaghegh,
Amantay Abdurakhmanov
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1880/1/012028
Subject(s) - character (mathematics) , representation (politics) , anomaly (physics) , anomaly detection , computer science , consistency (knowledge bases) , artificial intelligence , data mining , pattern recognition (psychology) , mathematics , physics , geometry , politics , political science , law , condensed matter physics
This paper proposes a character-level representation of unsupervised text data sets for anomaly detection problems. An empirical examination of the character-level text representation was conducted to demonstrate the ability to separate outlying and normal records using an ensemble of multiple classic numerical anomaly classifiers. Experimental results obtained on two different data sets confirmed the applicability of the developed unsupervised model to detect outlying instances in various real-world scenarios, providing the opportunity to quickly assess a large amount of textual data in terms of information consistency and conformity without knowledge of the data content itself.