English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Factors that limit the size of the input and output of a neural networkinclude memory requirements for the network states/activations to computegradients, as well as memory for the convolutional kernels or other weights.The memory restriction is especially limiting for applications where we want tolearn how to map volumetric data to the desired output, such as video-to-video.Recently developed fully reversible neural networks enable gradientcomputations using storage of the network states for a couple of layers only.While this saves a tremendous amount of memory, it is the convolutional kernelsthat take up most memory if fully reversible networks contain multipleinvertible pooling/coarsening layers. Invertible coarsening operators such asthe orthogonal wavelet transform cause the number of channels to growexplosively. We address this issue by combining fully reversible networks withlayers that contain the convolutional kernels in a compressed form directly.Specifically, we introduce a layer that has a symmetric block-low-rankstructure. In spirit, this layer is similar to bottleneck andsqueeze-and-expand structures. We contribute symmetry by construction, and acombination of notation and flattening of tensors allows us to interpret thesenetwork structures in linear algebraic fashion as a block-low-rank matrix infactorized form and observe various properties. A video segmentation exampleshows that we can train a network to segment the entire video in one go, whichwould not be possible, in terms of memory requirements, using non-reversiblenetworks and previously proposed reversible networks.

Symmetric block-low-rank layers for fully reversible multilevel neural networks