Premium
Uncertainty quantification for multilabel text classification
Author(s) -
Chen Wenshi,
Zhang Bowen,
Lu Mingyu
Publication year - 2020
Publication title -
wiley interdisciplinary reviews: data mining and knowledge discovery
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.506
H-Index - 47
eISSN - 1942-4795
pISSN - 1942-4787
DOI - 10.1002/widm.1384
Subject(s) - uncertainty quantification , computer science , artificial intelligence , machine learning , bayesian probability , bayesian network , artificial neural network , sensitivity (control systems) , data mining , engineering , electronic engineering
Abstract Deep neural networks have recently achieved impressive performance on multilabel text classification. However, the uncertainty in multilabel text classification tasks and their application in the model are often overlooked. To better understand and evaluate the uncertainty in multilabel text classification tasks, we propose a general framework called Uncertainty Quantification for Multilabel Text Classification framework. Based on the prediction results produced by traditional neural networks, the aleatory uncertainty of each classification label and the epistemic uncertainty of the prediction result can further be obtained by this framework. We design experiments to characterize the properties of aleatory uncertainty and epistemic uncertainty from the data characteristics and model features. The experimental results show that this framework is reasonable. Furthermore, we demonstrate how this framework allows us to define the model optimization criterion to identify policies that balance the expected training cost, model performance, and uncertainty sensitivity. This article is categorized under: Algorithmic Development > Bayesian Models