English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Cross-modal pre-training has shown impressive performance on a wide range ofdownstream tasks, benefiting from massive image-text pairs collected from theInternet. In practice, online data are growing constantly, highlighting theimportance of the ability of pre-trained model to learn from data that iscontinuously growing. Existing works on cross-modal pre-training mainly focuson training a network with fixed architecture. However, it is impractical tolimit the model capacity when considering the continuously growing nature ofpre-training data in real-world applications. On the other hand, it isimportant to utilize the knowledge in the current model to obtain efficienttraining and better performance. To address the above issues, in this paper, wepropose GrowCLIP, a data-driven automatic model growing algorithm forcontrastive language-image pre-training with continuous image-text pairs asinput. Specially, we adopt a dynamic growth space and seek out the optimalarchitecture at each growth step to adapt to online learning scenarios. And theshared encoder is proposed in our growth space to enhance the degree ofcross-modal fusion. Besides, we explore the effect of growth in differentdimensions, which could provide future references for the design of cross-modalmodel architecture. Finally, we employ parameter inheriting with momentum (PIM)to maintain the previous knowledge and address the issue of the local minimumdilemma. Compared with the existing methods, GrowCLIP improves 2.3% averagetop-1 accuracy on zero-shot image classification of 9 downstream tasks. As forzero-shot image retrieval, GrowCLIP can improve 1.2% for top-1 image-to-textrecall on Flickr30K dataset.

GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training