English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

During language acquisition, children follow a typical sequence of learningstages, whereby they first learn to categorize phonemes before they developtheir lexicon and eventually master increasingly complex syntactic structures.However, the computational principles that lead to this learning trajectoryremain largely unknown. To investigate this, we here compare the learningtrajectories of deep language models to those of children. Specifically, wetest whether, during its training, GPT-2 exhibits stages of languageacquisition comparable to those observed in children aged between 18 months and6 years. For this, we train 48 GPT-2 models from scratch and evaluate theirsyntactic and semantic abilities at each training step, using 96 probes curatedfrom the BLiMP, Zorro and BIG-Bench benchmarks. We then compare theseevaluations with the behavior of 54 children during language production. Ouranalyses reveal three main findings. First, similarly to children, the languagemodels tend to learn linguistic skills in a systematic order. Second, thislearning scheme is parallel: the language tasks that are learned last improvefrom the very first training steps. Third, some - but not all - learning stagesare shared between children and these language models. Overall, these resultsshed new light on the principles of language acquisition, and highlightimportant divergences in how humans and modern algorithms learn to processnatural language.

Language acquisition: do children and language models follow similar learning stages?