Open AccessA Survey of Scaling Distributed System Via Machine Learning and An Insight on Hadoop and Spark
Author(s) -
Atheel Sabih Shaker
Publication year - 2020
Publication title -
iop conference series. materials science and engineering
Language(s) - English
Resource type - Journals
eISSN - 1757-899X
pISSN - 1757-8981
DOI - 10.1088/1757-899x/928/3/032008
Subject(s) - spark (programming language) , big data , computer science , scaling , key (lock) , distributed learning , artificial intelligence , machine learning , distributed database , distributed computing , data mining , operating system , psychology , pedagogy , geometry , mathematics , programming language
This survey present and discuss distributed computing framework and distributed machine learning with evaluating the results of 10 papers on machine learning and big data. The first half discussed distributed computing framework, Hadoop and Spark. We briefly explained each structure and compared the key features between them. The second half consists of the survey in distributed machine learning. We briefly described the representative techniques as well as popular frameworks and discussed the major problem and challenges behind them used in different papers for scaling of distributed system via machine learning and big data techniques.