English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

A wide body of evidence shows that human language processing difficulty ispredicted by the information-theoretic measure surprisal, a word's negative logprobability in context. However, it is still unclear how to best estimate theseprobabilities needed for predicting human processing difficulty -- while along-standing belief held that models with lower perplexity would provide moreaccurate estimates of word predictability, and therefore lead to better readingtime predictions, recent work has shown that for very large models,psycholinguistic predictive power decreases. One reason could be that languagemodels might be more confident of their predictions than humans, because theyhave had exposure to several magnitudes more data. In this paper, we test whateffect temperature-scaling of large language model (LLM) predictions has onsurprisal estimates and their predictive power of reading times of Englishtexts. Firstly, we show that calibration of large language models typicallyimproves with model size, i.e. poorer calibration cannot account for poorer fitto reading times. Secondly, we find that temperature-scaling probabilities leadto a systematically better fit to reading times (up to 89% improvement in deltalog likelihood), across several reading time corpora. Finally, we show thatthis improvement in fit is chiefly driven by words that are composed ofmultiple subword tokens.

Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?