Open AccessMeteorological Application Based on Big Data Streaming Computation Method
Author(s) -
Xu Wang,
Hongyan Cheng,
Xiaoyan Liu,
Xiaojie Wang
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1533/4/042039
Subject(s) - computer science , big data , random forest , pruning , computation , data mining , decision tree , streaming data , tree (set theory) , algorithm , machine learning , mathematics , mathematical analysis , agronomy , biology
Big data analysis in the form of streaming computation has always been a problem to be solved at present, with relatively few research results and practical experience. The random forest method is currently the most extensively used classification algorithm. However, in the application scenario of streaming computation, real-time, volatility, and disorder features presented by data will lead to a gradual reduction in the accuracy of the algorithm. In this paper, the characteristics of the random forest algorithm are analyzed, and the idea of random forest pruning based on the accuracy of the decision tree is proposed. Meanwhile, to adapt to the changes in meteorological data, the concept of accuracy interval is combined to propose a method for the generation, verification, and supplementation of a new decision tree. Finally, a random forest that can be constantly updated with data is established to meet the requirements of the streaming big data environment for the algorithm. Actual meteorological data are used to verify the feasibility of the improved method. The results show that the new method has a higher classification accuracy in the real streaming big data scenario.