Open AccessCombined Processing of Hyperspectral and Thermal Images of Plants in Soil for the Early Diagnosis of Drought
Author(s) -
Maxim Lysov,
Konstantin Konstantinovich Pukhkii,
Вадим Турлапов
Publication year - 2021
Language(s) - English
Resource type - Conference proceedings
DOI - 10.20948/graphicon-2021-3027-529-541
Subject(s) - hyperspectral imaging , perceptron , artificial intelligence , normalized difference vegetation index , cluster analysis , artificial neural network , computer science , pattern recognition (psychology) , mean squared error , pixel , remote sensing , multilayer perceptron , environmental science , mathematics , statistics , geography , geology , climate change , oceanography
The possibilities of explainable artificial intelligence (XAI) in the early diagnosis of drought in plants based on hyperspectral images (HSI) are investigated. To provide the explainability and high accuracy to the result, we used the markup of HSI by superimposed Thermal IR (TIR) images of the last day of the experiment. Traditional HSI-based NDVI (Normalized Difference Vegetation Index) images were also constructed. The markup of HSIs based on their clustering by the k-means method into 5 classes was also objectified: wet plants; plants in a state of drought; wet soil; dry soil; background. For HSI, on the day of the experiment started, the number of clusters was set to 2 less to reflect the absence of drought circumstances. For use in training and testing, all HSIs channels are marked up with the results of clustering. The HIS-TIR-combination made it possible to determine the temperature for each plant pixel in HSI, and as the result to determine the number of days without watering. A fully connected Double Layer Perceptron (DLP) neural network was used to solve classification and regression problems. The trained DLP-regressor showed the average accuracy of predicting the temperature of plants on the control days of the experiment RMSE = 0.52 degrees, providing an error in predicting the day of the beginning of the drought for near 2 days. The DLP-classifier was able to classify the drought of the plant in the early stages (the fifth day) with an accuracy of 97.3%. Software tools: pytorch, scikit-learn, pysptools.