English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Transformer-based large language models are trained to make predictions aboutthe next word by aggregating representations of previous tokens through theirself-attention mechanism. In the field of cognitive modeling, such attentionpatterns have recently been interpreted as embodying the process of cue-basedretrieval, in which attention over multiple targets is taken to generateinterference and latency during retrieval. Under this framework, this workfirst defines an entropy-based predictor that quantifies the diffuseness ofself-attention, as well as distance-based predictors that capture theincremental change in attention patterns across timesteps. Moreover, followingrecent studies that question the informativeness of attention weights, we alsoexperiment with alternative methods for incorporating vector norms intoattention weights. Regression experiments using predictors calculated from theGPT-2 language model show that these predictors deliver a substantially betterfit to held-out self-paced reading and eye-tracking data over a rigorousbaseline including GPT-2 surprisal. Additionally, the distance-based predictorsgenerally demonstrated higher predictive power, with effect sizes of up to 6.59ms per standard deviation on self-paced reading times (compared to 2.82 ms forsurprisal) and 1.05 ms per standard deviation on eye-gaze durations (comparedto 3.81 ms for surprisal).

Entropy- and Distance-Based Predictors From GPT-2 Attention Patterns Predict Reading Times Over and Above GPT-2 Surprisal