English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Tremendous progress has been made in sequential processing with the recentadvances in recurrent neural networks. However, recurrent architectures facethe challenge of exploding/vanishing gradients during training, and requiresignificant computational resources to execute back-propagation through time.Moreover, large models are typically needed for executing complex sequentialtasks. To address these challenges, we propose a novel neuron model that hascosine activation with a time varying component for sequential processing. Theproposed neuron provides an efficient building block for projecting sequentialinputs into spectral domain, which helps to retain long-term dependencies withminimal extra model parameters and computation. A new type of recurrent networkarchitecture, named Oscillatory Fourier Neural Network, based on the proposedneuron is presented and applied to various types of sequential tasks. Wedemonstrate that recurrent neural network with the proposed neuron model ismathematically equivalent to a simplified form of discrete Fourier transformapplied onto periodical activation. In particular, the computationallyintensive back-propagation through time in training is eliminated, leading tofaster training while achieving the state of the art inference accuracy in adiverse group of sequential tasks. For instance, applying the proposed model tosentiment analysis on IMDB review dataset reaches 89.4% test accuracy within 5epochs, accompanied by over 35x reduction in the model size compared to LSTM.The proposed novel RNN architecture is well poised for intelligent sequentialprocessing in resource constrained hardware.

Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processing