KISS: an EBM-based approach for explaining deep models

The application of deep learning in data mining and knowledge extraction becomes more and more extensive, yet people often criticize deep models for its ”black box” trait. In this paper, we propose a simple algorithm for the model explanation by using the energy-based model theory and subset sampling. This approach can be model-agnostic without knowing the model architecture or be hybrid with the information inside the target model (e.g. the gradients). Through two image classification models, we compared our algorithm with other interpretation methods by testing the effects on the predictions and got an encouraging result.