English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the last decade, Convolutional Neural Network with a multi-layerarchitecture has advanced rapidly. However, training its complex network isvery space-consuming, since a lot of intermediate data are preserved acrosslayers, especially when processing high-dimension inputs with a big batch size.That poses great challenges to the limited memory capacity of currentaccelerators (e.g., GPUs). Existing efforts mitigate such bottleneck byexternal auxiliary solutions with additional hardware costs, and internalmodifications with potential accuracy penalty. Differently, our analysisreveals that computations intra- and inter-layers exhibit the spatial-temporalweak dependency and even complete independency features. That inspires us tobreak the traditional layer-by-layer (column) dataflow rule. Now operations arenovelly re-organized into rows throughout all convolution layers. Thislightweight design allows a majority of intermediate data to be removed withoutany loss of accuracy. We particularly study the weak dependency between twoconsecutive rows. For the resulting skewed memory consumption, we give twosolutions with different favorite scenarios. Evaluations on two representativenetworks confirm the effectiveness. We also validate that our middle dataflowoptimization can be smoothly embraced by existing works for better memoryreduction.

LR-CNN: Lightweight Row-centric Convolutional Neural Network Training for Memory Reduction