English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Finetuning a large vision language model (VLM) on a target dataset afterlarge scale pretraining is a dominant paradigm in visual question answering(VQA). Datasets for specialized tasks such as knowledge-based VQA or VQA in nonnatural-image domains are orders of magnitude smaller than those forgeneral-purpose VQA. While collecting additional labels for specialized tasksor domains can be challenging, unlabeled images are often available. Weintroduce SelTDA (Self-Taught Data Augmentation), a strategy for finetuninglarge VLMs on small-scale VQA datasets. SelTDA uses the VLM and target datasetto build a teacher model that can generate question-answer pseudolabelsdirectly conditioned on an image alone, allowing us to pseudolabel unlabeledimages. SelTDA then finetunes the initial VLM on the original dataset augmentedwith freshly pseudolabeled images. We describe a series of experiments showingthat our self-taught data augmentation increases robustness to adversariallysearched questions, counterfactual examples and rephrasings, improves domaingeneralization, and results in greater retention of numerical reasoning skills.The proposed strategy requires no additional annotations or architecturalmodifications, and is compatible with any modern encoder-decoder multimodaltransformer. Code available at https://github.com/codezakh/SelTDA.

Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images!