Open AccessKey Technologies of Large Data Stream System
Author(s) -
Dong Zhang
Publication year - 2021
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1881/4/042088
Subject(s) - computer science , data stream mining , data stream , real time computing , task (project management) , big data , scheduling (production processes) , data flow diagram , key (lock) , data management , distributed computing , process (computing) , data processing , data mining , database , systems engineering , engineering , telecommunications , computer security , operations management , operating system
In general, a large number of data streams are regarded as a continuous, continuous and infinite data series. It is used in network monitoring, sensor network, aerospace, meteorological measurement and control, financial services and other fields. Stream computing has the characteristics of large data scale, continuous, fast and disordered data arrival, easy data loss and diverse data processing. In view of these characteristics, there are new challenges in resource scheduling, data distribution and fault tolerance in task management of flow computing. This paper uses multi-layer Association big data rules to process the data wheel. The experimental results show that compared with the traditional task management method, the proposed method can provide the efficiency and accuracy of task management.