English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While large pre-trained language models (LMs) find greater use across NLP,existing evaluation protocols do not consider how LM language use aligns withparticular human demographic groups, which can be an important consideration inconversational AI applications. To remedy this gap, we consider how LM languageskills can be measured and compared to human sub-populations. We suggestclinical techniques from Speech Language Pathology, which has well-establishednorms for acquisition of language skills, organized by (human) age. We conductevaluation with a domain expert (i.e., a clinically licensed speech languagepathologist), and also propose automated techniques to substitute clinicalevaluation at scale. We find LM capability varies widely depending on task withGPT-3.5 mimicking the ability of a typical 6-9 year old at tasks requiringinference about word meanings and simultaneously outperforming a typical 21year old at memorization. GPT-3.5 (InstructGPT) also has trouble with sociallanguage use, exhibiting less than 50\% of the tested pragmatic skills. Itshows errors in understanding particular word parts-of-speech and associativeword relations, among other lexical features. Ultimately, findings reiteratethe importance of considering demographic alignment and conversational goalswhen using these models as public-facing tools. Our framework will be publiclyavailable via code, data, and a python package.

How Old is GPT?: The HumBEL Framework for Evaluating Language Models using Human Demographic Data