English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The use of neural language models to model human behavior has met with mixedsuccess. While some work has found that the surprisal estimates from thesemodels can be used to predict a wide range of human neural and behavioralresponses, other work studying more complex syntactic phenomena has found thatthese surprisal estimates generate incorrect behavioral predictions. This paperexplores the extent to which the misalignment between empirical andmodel-predicted behavior can be minimized by training models on moredevelopmentally plausible data, such as in the BabyLM Challenge. We trainedteacher language models on the BabyLM "strict-small" dataset and used sentencelevel surprisal estimates from these teacher models to create a curriculum. Wefound tentative evidence that our curriculum made it easier for models toacquire linguistic knowledge from the training data: on the subset of tasks inthe BabyLM challenge suite evaluating models' grammatical knowledge of English,models first trained on the BabyLM data curriculum and then on a few randomlyordered training epochs performed slightly better than models trained onrandomly ordered epochs alone. This improved linguistic knowledge acquisitiondid not result in better alignment with human reading behavior, however: modelstrained on the BabyLM dataset (with or without a curriculum) generatedpredictions that were as misaligned with human behavior as models trained onlarger less curated datasets. This suggests that training on developmentallyplausible datasets alone is likely insufficient to generate language modelscapable of accurately predicting human language processing.

Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?