English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we derive a PAC-Bayes bound on the generalisation gap, in asupervised time-series setting for a special class of discrete-time non-lineardynamical systems. This class includes stable recurrent neural networks (RNN),and the motivation for this work was its application to RNNs. In order toachieve the results, we impose some stability constraints, on the allowedmodels. Here, stability is understood in the sense of dynamical systems. ForRNNs, these stability conditions can be expressed in terms of conditions on theweights. We assume the processes involved are essentially bounded and the lossfunctions are Lipschitz. The proposed bound on the generalisation gap dependson the mixing coefficient of the data distribution, and the essential supremumof the data. Furthermore, the bound converges to zero as the dataset sizeincreases. In this paper, we 1) formalize the learning problem, 2) derive aPAC-Bayesian error bound for such systems, 3) discuss various consequences ofthis error bound, and 4) show an illustrative example, with discussions oncomputing the proposed bound. Unlike other available bounds the derived boundholds for non i.i.d. data (time-series) and it does not grow with the number ofsteps of the RNN.

PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNs