English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

It is well noted that coordinate-based MLPs benefit -- in terms of preservinghigh-frequency information -- through the encoding of coordinate positions asan array of Fourier features. Hitherto, the rationale for the effectiveness ofthese positional encodings has been mainly studied through a Fourier lens. Inthis paper, we strive to broaden this understanding by showing that alternativenon-Fourier embedding functions can indeed be used for positional encoding.Moreover, we show that their performance is entirely determined by a trade-offbetween the stable rank of the embedded matrix and the distance preservationbetween embedded coordinates. We further establish that the now ubiquitousFourier feature mapping of position is a special case that fulfills theseconditions. Consequently, we present a more general theory to analyzepositional encoding in terms of shifted basis functions. In addition, we arguethat employing a more complex positional encoding -- that scales exponentiallywith the number of modes -- requires only a linear (rather than deep)coordinate function to achieve comparable performance. Counter-intuitively, wedemonstrate that trading positional embedding complexity for network deepnessis orders of magnitude faster than current state-of-the-art; despite theadditional embedding complexity. To this end, we develop the necessarytheoretical formulae and empirically verify that our theoretical claims hold inpractice.

Trading Positional Complexity vs. Deepness in Coordinate Networks