English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Fine-tuning (via methods such as instruction-tuning or reinforcement learningfrom human feedback) is a crucial step in training language models to robustlycarry out tasks of interest. However, we lack a systematic understanding of theeffects of fine-tuning, particularly on tasks outside the narrow fine-tuningdistribution. In a simplified scenario, we demonstrate that improvingperformance on tasks within the fine-tuning data distribution comes at theexpense of suppressing model capabilities on other tasks. This degradation isespecially pronounced for tasks "closest" to the fine-tuning distribution. Wehypothesize that language models implicitly infer the task of the promptcorresponds, and the fine-tuning process predominantly skews this taskinference towards tasks in the fine-tuning distribution. To test thishypothesis, we propose Conjugate Prompting to see if we can recover pretrainedcapabilities. Conjugate prompting artificially makes the task look farther fromthe fine-tuning distribution while requiring the same capability. We find thatconjugate prompting systematically recovers some of the pretrainingcapabilities on our synthetic setup. We then apply conjugate prompting toreal-world LLMs using the observation that fine-tuning distributions aretypically heavily skewed towards English. We find that simply translating theprompts to different languages can cause the fine-tuned models to respond liketheir pretrained counterparts instead. This allows us to recover the in-contextlearning abilities lost via instruction tuning, and more concerningly, torecover harmful content generation suppressed by safety fine-tuning in chatbotslike ChatGPT.

Understanding Catastrophic Forgetting in Language Models via Implicit Inference