Open AccessBig data and remote sensing: A new software of ingestion
Author(s) -
B.E. Boudriki Semlali,
Chaker El Amrani
Publication year - 2021
Publication title -
international journal of power electronics and drive systems/international journal of electrical and computer engineering
Language(s) - English
Resource type - Journals
eISSN - 2722-2578
pISSN - 2722-256X
DOI - 10.11591/ijece.v11i2.pp1521-1530
Subject(s) - big data , computer science , preprocessor , software , data pre processing , satellite , data processing , remote sensing , real time computing , database , data mining , operating system , artificial intelligence , geography , engineering , aerospace engineering
Currently, remote sensing is widely used in environmental monitoring applications, mostly air quality mapping and climate change supervision. However, satellite sensors occur massive volumes of data in near-real-time, stored in multiple formats and are provided with high velocity and variety. Besides, the processing of satellite big data is challenging. Thus, this study aims to approve that satellite data are big data and proposes a new big data architecture for satellite data processing. The developed software is enabling an efficient remote sensing big data ingestion and preprocessing. As a result, the experiment results show that 86 percent of the unnecessary daily files are discarded with a data cleansing of 20 percent of the erroneous and inaccurate plots. The final output is integrated into the Hadoop system, especially the HDFS, HBase, and Hive, for extra calculation and processing.