English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large pre-trained multimodal models have demonstrated significant success ina range of downstream tasks, including image captioning, image-text retrieval,visual question answering (VQA), etc. However, many of these methods rely onimage-text pairs collected from the web as pre-training data and unfortunatelyoverlook the need for fine-grained feature alignment between vision andlanguage modalities, which requires detailed understanding of images andlanguage expressions. While integrating VQA and dense captioning (DC) intopre-training can address this issue, acquiring image-question-answer as well asimage-location-caption triplets is challenging and time-consuming.Additionally, publicly available datasets for VQA and dense captioning aretypically limited in scale due to manual data collection and labeling efforts.In this paper, we propose a novel method called Joint QA and DC GEneration(JADE), which utilizes a pre-trained multimodal model and easily-crawledimage-text pairs to automatically generate and filter large-scale VQA and densecaptioning datasets. We apply this method to the Conceptual Caption (CC3M)dataset to generate a new dataset called CC3M-QA-DC. Experiments show that whenused for pre-training in a multi-task manner, CC3M-QA-DC can improve theperformance with various backbones on various downstream tasks. Furthermore,our generated CC3M-QA-DC can be combined with larger image-text datasets (e.g.,CC15M) and achieve competitive results compared with models using much moredata. Code and dataset are available athttps://github.com/johncaged/OPT_Questioner.

Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner