English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The field of transparent Machine Learning (ML) has contributed many novelmethods aiming at better interpretability for computer vision and ML models ingeneral. But how useful the explanations provided by transparent ML methods arefor humans remains difficult to assess. Most studies evaluate interpretabilityin qualitative comparisons, they use experimental paradigms that do not allowfor direct comparisons amongst methods or they report only offline experimentswith no humans in the loop. While there are clear advantages of evaluationswith no humans in the loop, such as scalability, reproducibility and lessalgorithmic bias than with humans in the loop, these metrics are limited intheir usefulness if we do not understand how they relate to other metrics thattake human cognition into account. Here we investigate the quality ofinterpretable computer vision algorithms using techniques from psychophysics.In crowdsourced annotation tasks we study the impact of differentinterpretability approaches on annotation accuracy and task time. In order torelate these findings to quality measures for interpretability without humansin the loop we compare quality metrics with and without humans in the loop. Ourresults demonstrate that psychophysical experiments allow for robust qualityassessment of transparency in machine learning. Interestingly the qualitymetrics computed without humans in the loop did not provide a consistentranking of interpretability methods nor were they representative for how usefulan explanation was for humans. These findings highlight the potential ofmethods from classical psychophysics for modern machine learning applications.We hope that our results provide convincing arguments for evaluatinginterpretability in its natural habitat, human-ML interaction, if the goal isto obtain an authentic assessment of interpretability.

A psychophysics approach for quantitative comparison of interpretable computer vision models