English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper investigates the initialization bias of the Fourier neuraloperator (FNO). A mean-field theory for FNO is established, analyzing thebehavior of the random FNO from an ``edge of chaos'' perspective. We uncoverthat the forward and backward propagation behaviors exhibit characteristicsunique to FNO, induced by mode truncation, while also showcasing similaritiesto those of densely connected networks. Building upon this observation, we alsopropose a FNO version of the He initialization scheme to mitigate the negativeinitialization bias leading to training instability. Experimental resultsdemonstrate the effectiveness of our initialization scheme, enabling stabletraining of a 32-layer FNO without the need for additional techniques orsignificant performance degradation.

Initialization Bias of Fourier Neural Operator: Revisiting the Edge of Chaos