Open AccessBig Data Streaming Platforms: A Review
Author(s) -
Harish Kumar,
Ping Jack Soh,
Mohd Arfian Ismail
Publication year - 2022
Publication title -
iraqi journal for computer science and mathematics
Language(s) - English
Resource type - Journals
eISSN - 2958-0544
pISSN - 2788-7421
DOI - 10.52866/ijcsm.2022.02.01.010
Subject(s) - big data , computer science , data stream mining , data science , cloud computing , yesterday , usable , volume (thermodynamics) , stream processing , implementation , data stream , world wide web , distributed computing , data mining , software engineering , telecommunications , operating system , physics , quantum mechanics , astronomy
Yesterday’s “Big Data” is today’s “data.” As technology advances, new difficulties and new solutions emerge. In recent years, as a result of the development of Internet of Things (IoT) applications, the area of Data Mining has been confronted with the difficulty of analyzing and interpreting data streams in real time and at a high data throughput. This situation is known as the velocity element of big data. The rapid advancement of technology has come with an increased use of social media, computer networks, cloud computing, and the IoT. Experiments in the laboratory also generate a large quantity of data, which must be gathered, handled, and evaluated. This massive amount of data is referred to as “Big Data.” Analysts have seen an upsurge in data including valuable and worthless elements. In extracting usable information, data warehouses struggle to keep up with the rising volume of data collected. This article provides an overview of big data architecture and platforms, tools for data stream processing, and examples of implementations. Streaming computing is the focus of our project, which is building a data stream management system to deliver large-scale, cost-effective big data services. Owing to this study, the feasibility of large-scale data processing for distributed, real-time computing is improved even when the systems are overwhelmed.