A Collaborative Environment for Web Crawling and Web Data Analysis in ENEAGRID

In this document we provide an overview on the development and the integration in ENEAGRID of some tools able to download data from Web (Web Crawling), manage and display a large amount of data (Big Data), and extract from data relevant hidden information (Data Mining). We collected all these instruments inside the so called Web Crawling Project. Further, the corresponding environment, called Virtual Laboratory, is able to offer the possibility to use all these integrated tools remotely and by a simple graphical interface. A detailed description of the developed web application will be illustrated. Finally, some experimental results on the behaviour of the Web Crawling tool will be reported.