English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large Multimodal Models (LMM) are built across modalities and themisalignment between two modalities can result in "hallucination", generatingtextual outputs that are not grounded by the multimodal information in context.To address the multimodal misalignment issue, we adapt the ReinforcementLearning from Human Feedback (RLHF) from the text domain to the task ofvision-language alignment, where human annotators are asked to compare tworesponses and pinpoint the more hallucinated one, and the vision-language modelis trained to maximize the simulated human rewards. We propose a new alignmentalgorithm called Factually Augmented RLHF that augments the reward model withadditional factual information such as image captions and ground-truthmulti-choice options, which alleviates the reward hacking phenomenon in RLHFand further improves the performance. We also enhance the GPT-4-generatedtraining data (for vision instruction tuning) with previously availablehuman-written image-text pairs to improve the general capabilities of ourmodel. To evaluate the proposed approach in real-world scenarios, we develop anew evaluation benchmark MMHAL-BENCH with a special focus on penalizinghallucinations. As the first LMM trained with RLHF, our approach achievesremarkable improvement on the LLaVA-Bench dataset with the 94% performancelevel of the text-only GPT-4 (while previous best methods can only achieve the87% level), and an improvement by 60% on MMHAL-BENCH over other baselines. Weopensource our code, model, data at https://llava-rlhf.github.io.

Aligning Large Multimodal Models with Factually Augmented RLHF