English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large language models (LLMs) have shown remarkable performance in naturallanguage processing (NLP) tasks. To comprehend and execute diverse humaninstructions over image data, instruction-tuned large vision-language models(LVLMs) have been introduced. However, LVLMs may suffer from different types ofobject hallucinations. Nevertheless, LVLMs are evaluated for coarse-grainedobject hallucinations only (i.e., generated objects non-existent in the inputimage). The fine-grained object attributes and behaviors non-existent in theimage may still be generated but not measured by the current evaluationmethods. In this paper, we thus focus on reducing fine-grained hallucinationsof LVLMs. We propose \textit{ReCaption}, a framework that consists of twocomponents: rewriting captions using ChatGPT and fine-tuning theinstruction-tuned LVLMs on the rewritten captions. We also propose afine-grained probing-based evaluation method named \textit{Fine-Grained ObjectHallucination Evaluation} (\textit{FGHE}). Our experiment results demonstratethat ReCaption effectively reduces fine-grained object hallucination fordifferent LVLM options and improves their text generation quality. The code canbe found at https://github.com/Anonymousanoy/FOHE.

Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewrites