Open AccessGenerating visual explanations with natural language
Author(s) -
Hendricks Lisa Anne,
Rohrbach Anna,
Schiele Bernt,
Darrell Trevor,
Akata Zeynep
Publication year - 2021
Publication title -
applied ai letters
Language(s) - English
Resource type - Journals
ISSN - 2689-5595
DOI - 10.1002/ail2.55
Subject(s) - discriminative model , phrase , meaning (existential) , class (philosophy) , natural (archaeology) , task (project management) , natural language , computer science , artificial intelligence , linguistics , image (mathematics) , psychology , natural language processing , cognitive psychology , philosophy , history , management , archaeology , economics , psychotherapist
We generate natural language explanations for a fine‐grained visual recognition task. Our explanations fulfill two criteria. First, explanations are class discriminative , meaning they mention attributes in an image which are important to identify a class. Second, explanations are image relevant , meaning they reflect the actual content of an image. Our system, composed of an explanation sampler and phrase‐critic model, generates class discriminative and image relevant explanations. In addition, we demonstrate that our explanations can help humans decide whether to accept or reject an AI decision.