English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The goal of this paper is to explore how Transformer language models processsemantic knowledge, especially regarding the plausibility of noun-verbrelations. First, I demonstrate GPT2 exhibits a higher degree of similaritywith humans in plausibility processing compared to other Transformer languagemodels. Next, I delve into how knowledge of plausibility is contained withinattention heads of GPT2 and how these heads causally contribute to GPT2'splausibility processing ability. Through several experiments, it was foundthat: i) GPT2 has a number of attention heads that detect plausible noun-verbrelationships; ii) these heads collectively contribute to the Transformer'sability to process plausibility, albeit to varying degrees; and iii) attentionheads' individual performance in detecting plausibility does not necessarilycorrelate with how much they contribute to GPT2's plausibility processingability.

Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT