English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Equivariant Transformers such as Equiformer have demonstrated the efficacy ofapplying Transformers to the domain of 3D atomistic systems. However, they arelimited to small degrees of equivariant representations due to theircomputational complexity. In this paper, we investigate whether thesearchitectures can scale well to higher degrees. Starting from Equiformer, wefirst replace $SO(3)$ convolutions with eSCN convolutions to efficientlyincorporate higher-degree tensors. Then, to better leverage the power of higherdegrees, we propose three architectural improvements -- attentionre-normalization, separable $S^2$ activation and separable layer normalization.Putting this all together, we propose EquiformerV2, which outperforms previousstate-of-the-art methods on large-scale OC20 dataset by up to $9\%$ on forces,$4\%$ on energies, offers better speed-accuracy trade-offs, and $2\times$reduction in DFT calculations needed for computing adsorption energies.Additionally, EquiformerV2 trained on only OC22 dataset outperforms GemNet-OCtrained on both OC20 and OC22 datasets, achieving much better data efficiency.Finally, we compare EquiformerV2 with Equiformer on QM9 and OC20 S2EF-2Mdatasets to better understand the performance gain brought by higher degrees.

EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations