Open AccessDesign and Implementation of Hotel Big Data Analysis Platform Based on Hadoop and Spark
Author(s) -
Dianwei Chi,
Chunhua Tang,
Yin Chen
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/2010/1/012079
Subject(s) - big data , spark (programming language) , upload , computer science , web crawler , database , data pre processing , data warehouse , preprocessor , world wide web , operating system , data mining , artificial intelligence , programming language
In order to provide decision support for user travel and hotel management planning, this paper proposes the design and implementation of a hotel big data analysis platform based on Hadoop and Spark technology, using hotel basic data and user comment data as data sources. First, we design a crawler program to collect basic data and user comment data of city hotels in Qingdao, Shandong Province, and upload them to the hadoop storage platform, use Spark for big data preprocessing, use Hive to build a big data platform data warehouse, and finally use echarts to analyze the results and make decisions perform a visual display.