English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep learning models can encounter unexpected failures, especially whendealing with challenging sub-populations. One common reason for these failuresis the occurrence of objects in backgrounds that are rarely seen duringtraining. To gain a better understanding of these failure modes,human-interpretable descriptions are crucial for further analysis andimprovement which is expensive. In this study, we propose an end-to-endframework that utilizes the capabilities of large language models (ChatGPT) andvision-language deep models (CLIP) to generate text descriptions of failuremodes associated with spurious correlations (e.g. rarely seen backgrounds)without human-in-the-loop intervention. These descriptions can be used togenerate synthetic data using generative models, such as diffusion models. Themodel can now use this generated data to learn from its weaknesses and enhanceits performance on backgrounds that are uncommon for each class of data. Ourapproach serves as a broad solution, promising progress in comprehending modelfailure modes and strengthening deep learning models across a wide range offailure scenarios (e.g. bacckgrounds, colors) automatically in a few-shotmanner. Our experiments have shown remarkable \textbf{improvements in accuracy($\sim \textbf{21%}$)} on hard sub-populations (particularly for wrongbackground association) across $40$ different models, such as ResNets,EfficientNets, DenseNets, Vision Transformer (ViT), SwAVs, MoCos, DINOs, andCLIPs on various datasets such as ImageNet-1000, CIFAR-10, and CIFAR-100.

Identifying and Mitigating Model Failures through Few-shot CLIP-aided Diffusion Generation