Open AccessAnalysis of multi-class classification performance metrics for remote sensing imagery imbalanced datasets
Author(s) -
Andrea González-Ramírez,
Josue A. Lopez,
D. Torres,
Israel Yañez-Vargas
Publication year - 2021
Publication title -
revista de análisis cuantitativo y estadístico
Language(s) - English
Resource type - Journals
ISSN - 2410-3438
DOI - 10.35429/jqsa.2021.22.8.11.17
Subject(s) - support vector machine , confusion matrix , computer science , metric (unit) , confusion , artificial intelligence , classifier (uml) , binary classification , correlation , machine learning , pattern recognition (psychology) , matthews correlation coefficient , class (philosophy) , performance metric , binary number , set (abstract data type) , data mining , mathematics , psychology , operations management , geometry , management , arithmetic , psychoanalysis , economics , programming language
Remote sensing imaging datasets for classification generally present high levels of imbalance between classes of interest. This work presented a study of a set of performance evaluation metrics for an imbalance dataset. In this work, a support vector machine (SVM) was used to perform the classification of seven classes of interest in a popular dataset called Salinas-A. The performance evaluation of the classifier was performed using two types of metrics: 1) Metrics for multi-class classification, and 2) Metrics based on the binary confusion matrix. In the results, a comparison of the scores of each metric is developed, some being more optimistic than others due to the bias that they present given the imbalance. In addition, our case study helps to conclude that the Matthews correlation coefficient (MCC) presents the lowest bias in imbalanced cases and is regarded to be robust metric. These results can be extended to any imbalanced dataset taking into account the equations developed by Luque.