Application Research of Crawler and Data Analysis Based on Python | Zendy

Hejing Wu | Zendy; Liu Fang | Zendy; Long Zhao | Zendy; Yabin Shao | Zendy; Ran Cui | Zendy

AI Assistant Blog Pricing

Home ZAIA Blog

Open Access

Application Research of Crawler and Data Analysis Based on Python

Author(s) -

Hejing Wu,

Liu Fang,

Long Zhao,

Yabin Shao,

Ran Cui

Publication year - 2020

Publication title -

international journal of advanced network monitoring and controls

Language(s) - English

Resource type - Journals

ISSN - 2470-8038

DOI - 10.21307/ijanmc-2020-018

Subject(s) - web crawler , upload , python (programming language) , computer science , data mining , focused crawler , information retrieval , database , world wide web , operating system , the internet , static web page , web server

Combined with the actual situation, this paper explores how to develop a crawler method based on the specific framework for the complete interface of steam manufacturers and stores, which should be able to automatically and efficiently crawl the data of specific targets, analyze the dynamic pages, and complete the data cleaning, downloading, saving and other operations, explore the methods of general data analysis, and Analyze the downloaded data, extract useful information from it, analyze and summarize the specific crawler method and data analysis method through practical application. Keywords-Python; Scrapy; Selenium; BeautifulSoup

The content you want is available to Zendy users.

Already have an account? Click here to sign in.

Having issues? You can contact us here

Accelerating Research