English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Algorithmic generalization in machine learning refers to the ability to learnthe underlying algorithm that generates data in a way that generalizesout-of-distribution. This is generally considered a difficult task for mostmachine learning algorithms. Here, we analyze algorithmic generalization whencounting is required, either implicitly or explicitly. We show that standardTransformers are based on architectural decisions that hinderout-of-distribution performance for such tasks. In particular, we discuss theconsequences of using layer normalization and of normalizing the attentionweights via softmax. With ablation of the problematic operations, wedemonstrate that a modified transformer can exhibit a good algorithmicgeneralization performance on counting while using a very lightweightarchitecture.

Counting and Algorithmic Generalization with Transformers