English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Binary Balanced Tree RvNNs (BBT-RvNNs) enforce sequence composition accordingto a preset balanced binary tree structure. Thus, their non-linear recursiondepth is just $\log_2 n$ ($n$ being the sequence length). Such logarithmicscaling makes BBT-RvNNs efficient and scalable on long sequence tasks such asLong Range Arena (LRA). However, such computational efficiency comes at a costbecause BBT-RvNNs cannot solve simple arithmetic tasks like ListOps. On theflip side, RvNNs (e.g., Beam Tree RvNN) that do succeed on ListOps (and otherstructure-sensitive tasks like formal logical inference) are generally severaltimes more expensive than even RNNs. In this paper, we introduce a novelframework -- Recursion in Recursion (RIR) to strike a balance between the twosides - getting some of the benefits from both worlds. In RIR, we use a form oftwo-level nested recursion - where the outer recursion is a $k$-ary balancedtree model with another recursive model (inner recursion) implementing its cellfunction. For the inner recursion, we choose Beam Tree RvNNs (BT-RvNN). Toadjust BT-RvNNs within RIR we also propose a novel strategy of beam alignment.Overall, this entails that the total recursive depth in RIR is upper-bounded by$k \log_k n$. Our best RIR-based model is the first model that demonstrateshigh ($\geq 90\%$) length-generalization performance on ListOps while at thesame time being scalable enough to be trainable on long sequence inputs fromLRA. Moreover, in terms of accuracy in the LRA language tasks, it performscompetitively with Structured State Space Models (SSMs) without any specialinitialization - outperforming Transformers by a large margin. On the otherhand, while SSMs can marginally outperform RIR on LRA, they (SSMs) fail tolength-generalize on ListOps. Our code is available at:\url{https://github.com/JRC1995/BeamRecursionFamily/}.

Recursion in Recursion: Two-Level Nested Recursion for Length Generalization with Scalability