English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Visual instruction tuning large language model(LLM) on image-text pairs hasachieved general-purpose vision-language abilities. However, the lack ofregion-text pairs limits their advancements to fine-grained multimodalunderstanding. In this paper, we propose spatial instruction tuning, whichintroduces the reference to the region-of-interest(RoI) in the instruction.Before sending to LLM, the reference is replaced by RoI features andinterleaved with language embeddings as a sequence. Our model GPT4RoI, trainedon 7 region-text pair datasets, brings an unprecedented interactive andconversational experience compared to previous image-level models. (1)Interaction beyond language: Users can interact with our model by both languageand drawing bounding boxes to flexibly adjust the referring granularity. (2)Versatile multimodal abilities: A variety of attribute information within eachRoI can be mined by GPT4RoI, e.g., color, shape, material, action, etc.Furthermore, it can reason about multiple RoIs based on common sense. On theVisual Commonsense Reasoning(VCR) dataset, GPT4RoI achieves a remarkableaccuracy of 81.6%, surpassing all existing models by a significant margin (thesecond place is 75.6%) and almost reaching human-level performance of 85.0%.The code, dataset, and demo can be found athttps://github.com/jshilong/GPT4RoI.

GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest