Towards Extended Data Mining: An Examination of Technical Aspects

Data mining has been an active research area for a couple of decades, yet the complicated nature of data mining is still not fully understood. One common misunderstanding of data mining is: Give me the data set, and data mining tools will show me the hidden knowledge. However, this thinking is quite naive, and is not realistic in many real world applications. In this paper, we explore extended data mining, which has the ultimate goal of automatically collecting additional data when needed for effective data mining. Existing web crawling and scraping techniques can be incorporated, but additional steps are still needed. In this paper, we examine important technical aspects for extended data mining via web crawling and scraping.