Premium
Dynamic resource allocation for big data streams based on data characteristics (5 V s)
Author(s) -
Kaur Navroop,
Sood Sandeep K.
Publication year - 2017
Publication title -
international journal of network management
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.373
H-Index - 28
eISSN - 1099-1190
pISSN - 1055-7148
DOI - 10.1002/nem.1978
Subject(s) - computer science , data stream mining , streams , big data , resource (disambiguation) , resource allocation , data science , distributed computing , data mining , computer network
Summary Various Internet‐based applications such as social media, business transactions, mobile applications, cyber‐physical systems, and Internet of Things have led to the generation of big data streams in every field. The growing need to extract knowledge from big data streams has pioneered the challenge of selecting appropriate cloud resources. The current techniques allocate resources based on data characteristics. But because of the stochastic nature of data generation, the characteristics of data in big data streams are unknown. This poses difficulty in selecting and allocating appropriate resources to big data stream. Working towards this direction, this paper proposes a system that predicts the data characteristics in terms of volume, velocity, variety, variability, and veracity. The predicted values are expressed in a quadruple called C haracteristics o f B ig dat a (CoBa). Thereafter, the proposed system uses self‐organizing maps to dynamically create clusters of cloud resources. One of these clusters is allocated to the big data stream based on its CoBa. The proposed system is dynamic in the sense that it changes the cloud cluster allocated to big data stream if its CoBa changes. Experimental results show that the proposed system has a performance edge over other streaming data processing tools such as Storm, Flume, and S4.