English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Shortcut learning is common but harmful to deep learning models, leading todegenerated feature representations and consequently jeopardizing the model'sgeneralizability and interpretability. However, shortcut learning in the widelyused Vision Transformer framework is largely unknown. Meanwhile, introducingdomain-specific knowledge is a major approach to rectifying the shortcuts,which are predominated by background related factors. For example, in themedical imaging field, eye-gaze data from radiologists is an effective humanvisual prior knowledge that has the great potential to guide the deep learningmodels to focus on meaningful foreground regions of interest. However,obtaining eye-gaze data is time-consuming, labor-intensive and sometimes evennot practical. In this work, we propose a novel and effective saliency-guidedvision transformer (SGT) model to rectify shortcut learning in ViT with theabsence of eye-gaze data. Specifically, a computational visual saliency modelis adopted to predict saliency maps for input image samples. Then, the saliencymaps are used to distil the most informative image patches. In the proposedSGT, the self-attention among image patches focus only on the distilledinformative ones. Considering this distill operation may lead to globalinformation lost, we further introduce, in the last encoder layer, a residualconnection that captures the self-attention across all the image patches. Theexperiment results on four independent public datasets show that our SGTframework can effectively learn and leverage human prior knowledge without eyegaze data and achieves much better performance than baselines. Meanwhile, itsuccessfully rectifies the harmful shortcut learning and significantly improvesthe interpretability of the ViT model, demonstrating the promise oftransferring human prior knowledge derived visual saliency in rectifyingshortcut learning

Rectify ViT Shortcut Learning by Visual Saliency