English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite their widespread success in various domains, Transformer networkshave yet to perform well across datasets in the domain of 3D atomistic graphssuch as molecules even when 3D-related inductive biases like translationalinvariance and rotational equivariance are considered. In this paper, wedemonstrate that Transformers can generalize well to 3D atomistic graphs andpresent Equiformer, a graph neural network leveraging the strength ofTransformer architectures and incorporating SE(3)/E(3)-equivariant featuresbased on irreducible representations (irreps). First, we propose a simple andeffective architecture by only replacing original operations in Transformerswith their equivariant counterparts and including tensor products. Usingequivariant operations enables encoding equivariant information in channels ofirreps features without complicating graph structures. With minimalmodifications to Transformers, this architecture has already achieved strongempirical results. Second, we propose a novel attention mechanism calledequivariant graph attention, which improves upon typical attention inTransformers through replacing dot product attention with multi-layerperceptron attention and including non-linear message passing. With these twoinnovations, Equiformer achieves competitive results to previous models on QM9,MD17 and OC20 datasets.

Equiformer: Equivariant Graph Attention Transformer for 3D Atomistic Graphs