English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Achieving efficient and robust multi-channel data learning is a challengingtask in data science. By exploiting low-rankness in the transformed domain,i.e., transformed low-rankness, tensor Singular Value Decomposition (t-SVD) hasachieved extensive success in multi-channel data representation and hasrecently been extended to function representation such as Neural Networks witht-product layers (t-NNs). However, it still remains unclear how t-SVDtheoretically affects the learning behavior of t-NNs. This paper is the firstto answer this question by deriving the upper bounds of the generalizationerror of both standard and adversarially trained t-NNs. It reveals that thet-NNs compressed by exact transformed low-rank parameterization can achieve asharper adversarial generalization bound. In practice, although t-NNs rarelyhave exactly transformed low-rank weights, our analysis further shows that byadversarial training with gradient flow (GF), the over-parameterized t-NNs withReLU activations are trained with implicit regularization towards transformedlow-rank parameterization under certain conditions. We also establishadversarial generalization bounds for t-NNs with approximately transformedlow-rank weights. Our analysis indicates that the transformed low-rankparameterization can promisingly enhance robust generalization for t-NNs.

Transformed Low-Rank Parameterization Can Help Robust Generalization for Tensor Neural Networks