English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The instruction learning paradigm -- where a model learns to perform newtasks from task descriptions alone -- has become popular in general-purposemodel research. The capabilities of large transformer models as instructionlearners, however, remain poorly understood. We use a controlled syntheticenvironment to characterize such capabilities. Specifically, we use the task ofdeciding whether a given string matches a regular expression (viewed as aninstruction) to identify properties of tasks, instructions, and instances thatmake instruction learning challenging. For instance, we find that our model, afine-tuned T5-based text2text transformer, struggles with large regularlanguages, suggesting that less precise instructions are challenging formodels. Additionally, instruction executions that require tracking longercontexts of prior steps are also more difficult. We use our findings tosystematically construct a challenging instruction learning dataset, which wecall Hard RegSet. Fine-tuning on Hard RegSet, our large transformer learns tocorrectly interpret only 65.6% of test instructions (with at least 90%accuracy), and 11%-24% of the instructions in out-of-distributiongeneralization settings. We propose Hard RegSet as a challenging instructionlearning task, and a controlled environment for studying instruction learning.

What Makes Instruction Learning Hard? An Investigation and a New Challenge in a Synthetic Environment