English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Leveraging large language models (LLMs) to integrate off-the-shelf tools(e.g., visual models and image processing functions) is a promising researchdirection to build powerful visual assistants for solving diverse visual tasks.However, the learning capability is rarely explored in existing methods, asthey freeze the used tools after deployment, thereby limiting thegeneralization to new environments requiring specific knowledge. In this paper,we propose CLOVA, a Closed-LOop Visual Assistant to address this limitation,which encompasses inference, reflection, and learning phases in a closed-loopframework. During inference, LLMs generate programs and execute correspondingtools to accomplish given tasks. The reflection phase introduces a multimodalglobal-local reflection scheme to analyze whether and which tool needs to beupdated based on environmental feedback. Lastly, the learning phase uses threeflexible manners to collect training data in real-time and introduces a novelprompt tuning scheme to update the tools, enabling CLOVA to efficiently learnspecific knowledge for new environments without human involvement. Experimentsshow that CLOVA outperforms tool-usage methods by 5% in visual questionanswering and multiple-image reasoning tasks, by 10% in knowledge taggingtasks, and by 20% in image editing tasks, highlighting the significance of thelearning capability for general visual assistants.

CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update