English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

The ability of large language models (LLMs) to process visual inputs hasgiven rise to general-purpose vision systems, unifying various vision-language(VL) tasks by instruction tuning. However, due to the enormous diversity ininput-output formats in the vision domain, existing general-purpose models failto successfully integrate segmentation and multi-image inputs with coarse-leveltasks into a single framework. In this work, we introduce VistaLLM, a powerfulvisual system that addresses coarse- and fine-grained VL tasks over single andmultiple input images using a unified framework. VistaLLM utilizes aninstruction-guided image tokenizer that filters global embeddings using taskdescriptions to extract compressed and refined features from numerous images.Moreover, VistaLLM employs a gradient-aware adaptive sampling technique torepresent binary segmentation masks as sequences, significantly improving overpreviously used uniform sampling. To bolster the desired capability ofVistaLLM, we curate CoinIt, a comprehensive coarse-to-fine instruction tuningdataset with 6.8M samples. We also address the lack of multi-image groundingdatasets by introducing a novel task, AttCoSeg (Attribute-levelCo-Segmentation), which boosts the model's reasoning and grounding capabilityover multiple input images. Extensive experiments on a wide range of V- and VLtasks demonstrate the effectiveness of VistaLLM by achieving consistentstate-of-the-art performance over strong baselines across all downstream tasks.Our project page can be found at https://shramanpramanick.github.io/VistaLLM/.

Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model