English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The CLIP (Contrastive Language-Image Pre-training) model and its variants arebecoming the de facto backbone in many applications. However, training a CLIPmodel from hundreds of millions of image-text pairs can be prohibitivelyexpensive. Furthermore, the conventional CLIP model doesn't differentiatebetween the visual semantics and meaning of text regions embedded in images.This can lead to non-robustness when the text in the embedded region doesn'tmatch the image's visual appearance. In this paper, we discuss two effectiveapproaches to improve the efficiency and robustness of CLIP training: (1)augmenting the training dataset while maintaining the same number ofoptimization steps, and (2) filtering out samples that contain text regions inthe image. By doing so, we significantly improve the classification andretrieval accuracy on public benchmarks like ImageNet and CoCo. Filtering outimages with text regions also protects the model from typographic attacks. Toverify this, we build a new dataset named ImageNet with Adversarial TextRegions (ImageNet-Attr). Our filter-based CLIP model demonstrates a top-1accuracy of 68.78\%, outperforming previous models whose accuracy was all below50\%.

Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustness