English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large multimodal models (LMMs) suffer from multimodal hallucination, wherethey provide incorrect responses misaligned with the given visual information.Recent works have conjectured that one of the reasons behind multimodalhallucination might be due to the vision encoder failing to ground on the imageproperly. To mitigate this issue, we propose a novel approach that leveragesself-feedback as visual cues. Building on this approach, we introduce Volcano,a multimodal self-feedback guided revision model. Volcano generates naturallanguage feedback to its initial response based on the provided visualinformation and utilizes this feedback to self-revise its initial response.Volcano effectively reduces multimodal hallucination and achievesstate-of-the-art on MMHal-Bench, POPE, and GAVIE. It also improves on generalmultimodal abilities and outperforms previous models on MM-Vet and MMBench.Through a qualitative analysis, we show that Volcano's feedback is properlygrounded on the image than the initial response. This indicates that Volcanocan provide itself with richer visual information, helping alleviate multimodalhallucination. We publicly release Volcano models of 7B and 13B sizes alongwith the data and code at https://github.com/kaistAI/Volcano.

Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision