English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

We demonstrate that LLMs may learn indicators of document usefulness andmodulate their updates accordingly. We introduce random strings ("tags") asindicators of usefulness in a synthetic fine-tuning dataset. Fine-tuning onthis dataset leads to implicit meta-learning (IML): in further fine-tuning, themodel updates to make more use of text that is tagged as useful. We perform athorough empirical investigation of this phenomenon, finding (among otherthings) that (i) it occurs in both pretrained LLMs and those trained fromscratch, as well as on a vision task, and (ii) larger models and smaller batchsizes tend to give more IML. We also use probing to examine how IML changes theway models store knowledge in their parameters. Finally, we reflect on what ourresults might imply about capabilities, risks, and controllability of future AIsystems. Our code can be found athttps://github.com/krasheninnikov/internalization.

Implicit meta-learning may lead language models to trust more reliable sources