English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Graph Attention Networks (GATs) are one of the most popular GNN architecturesand are considered as the state-of-the-art architecture for representationlearning with graphs. In GAT, every node attends to its neighbors given its ownrepresentation as the query. However, in this paper we show that GAT computes avery limited kind of attention: the ranking of the attention scores isunconditioned on the query node. We formally define this restricted kind ofattention as static attention and distinguish it from a strictly moreexpressive dynamic attention. Because GATs use a static attention mechanism,there are simple graph problems that GAT cannot express: in a controlledproblem, we show that static attention hinders GAT from even fitting thetraining data. To remove this limitation, we introduce a simple fix bymodifying the order of operations and propose GATv2: a dynamic graph attentionvariant that is strictly more expressive than GAT. We perform an extensiveevaluation and show that GATv2 outperforms GAT across 11 OGB and otherbenchmarks while we match their parametric costs. Our code is available athttps://github.com/tech-srl/how_attentive_are_gats . GATv2 is available as partof the PyTorch Geometric library, the Deep Graph Library, and the TensorFlowGNN library.

How Attentive are Graph Attention Networks?