English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

One way to interpret the reasoning power of transformer-based language modelsis to describe the types of logical rules they can resolve over some inputtext. Recently, Chiang et al. (2023) showed that finite-precision transformerscan be equivalently expressed in a generalization of first-order logic.However, finite-precision transformers are a weak transformer variant because,as we show, a single head can only attend to a constant number of tokens and,in particular, cannot represent uniform attention. Since attending broadly is acore capability for transformers, we ask whether a minimally more expressivemodel that can attend universally can also be characterized in logic. To thisend, we analyze transformers whose forward pass is computed in $\log n$precision on contexts of length $n$. We prove that any log-precisiontransformer can be equivalently expressed as a first-order logic sentence that,in addition to standard universal and existential quantifiers, may also containmajority-vote quantifiers. This is the tightest known upper bound and firstlogical characterization of log-precision transformers.

A Logic for Expressing Log-Precision Transformers