English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Current LLMs have demonstrated remarkable capabilities in addressing users'requests for various types of information. However, these models are limited bythe most recent data available in their pretraining corpora, rendering themincapable of providing up-to-date information. Retraining LLMs from scratch iscost-prohibitive, and the effectiveness of continual fine-tuning on new corporahas not been thoroughly examined. Additionally, current update procedurestypically demand significant human input to prepare the information into morestructured format, such as knowledge triples, conversational data or responseswith human feedback. In this study, we conduct a comprehensive examination of anovel self information update task in LLMs, which only requires the provisionof informative text corpora. For instance, we can use the latest news articlesto update the LLMs' existing knowledge. We define the self information updatetask and assess the continual fine-tuning approach for this purpose. We observethat the naive method of continual fine-tuning can be problematic due to LLMs'exposure bias, which prioritizes existing information over new information weaim to integrate and leads to incorrect reasoning chains that ultimatelydiminish the efficacy of information updates. Based on our analysis, we proposean effective method to mitigate exposure bias by incorporating the selection ofrelevant facts into training losses. Furthermore, we develop a dataset toevaluate information updates, derived from news articles published after March2023. Experimental results demonstrate that our proposed approach significantlyincreases the factual consistency score (0 to 1) by 0.16 while having minimalimpact on performance for instructions not directly related to the newinformation.

Self Information Update for Large Language Models through Mitigating Exposure Bias