Open AccessPotential threats mining methods based on correlation analysis of multi‐type logs
Author(s) -
Qin Tao,
Gao Yuli,
Wei Lingyan,
Liu Zhaoli,
Wang Chenxu
Publication year - 2018
Publication title -
iet networks
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.466
H-Index - 21
ISSN - 2047-4962
DOI - 10.1049/iet-net.2017.0188
Subject(s) - computer science , normalization (sociology) , data mining , anomaly detection , database normalization , construct (python library) , correlation , process (computing) , artificial intelligence , pattern recognition (psychology) , mathematics , geometry , sociology , anthropology , programming language , operating system
Log analysis is an efficiency way to detect threats by scrutinizing the events recorded by the operating systems and devices. However, it is more and more difficult to discover threats accurately due to the massive amount of logs and their various formats. Focusing on this problem, the authors propose a method for potential threats mining based on the correlation analysis of multi‐type logs. Firstly, they extract 12 features, including behavior‐related, attribute‐related and measurable features, from multi‐type logs based on the characteristics of known and potential attacks. They also propose normalization method to deal with these heterogeneous features. Secondly, focusing on solving the problem that analyzing a single type of log can only detect some specific attacks, they employ the logistic regression model to perform correlation analysis on multi‐type logs. Finally, they construct an anomaly detection platform integrated with parallel processing mechanism to process the massive records. The experimental results based on logs collected show that the proposed method has high detection accuracy and low computational complexity, which can be applied to mine potential threats and abnormal users from the massive logs in an actual network environment.