English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The recent upsurge in pre-trained large models (e.g. GPT-4) has swept acrossthe entire deep learning community. Such powerful large language models (LLMs)demonstrate advanced generative ability and multimodal understandingcapability, which quickly achieve new state-of-the-art performances on avariety of benchmarks. The pre-trained LLM usually plays the role as auniversal AI model that can conduct various tasks, including context reasoning,article analysis and image content comprehension. However, considering theprohibitively high memory and computational cost for implementing such a largemodel, the conventional models (such as CNN and ViT), are still essential formany visual perception tasks. In this paper, we propose to enhance therepresentation ability of ordinary vision models for perception tasks (e.g.image classification) by taking advantage of large pre-trained models. Wepresent a new learning paradigm in which the knowledge extracted from largepre-trained models are utilized to help models like CNN and ViT learn enhancedrepresentations and achieve better performance. Firstly, we curate a highquality description set by prompting a multimodal LLM to generate descriptivetext for all training images. Furthermore, we feed these detailed descriptionsinto a pre-trained encoder to extract text embeddings with rich semanticinformation that encodes the content of images. During training, textembeddings will serve as extra supervising signals and be aligned with imagerepresentations learned by vision models. The alignment process helps visionmodels learn better and achieve higher accuracy with the assistance ofpre-trained LLMs. We conduct extensive experiments to verify that the proposedalgorithm consistently improves the performance for various vision models withheterogeneous architectures.

GPT4Image: Can Large Pre-trained Models Help Vision Models on Perception Tasks?