English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The computing cost and memory demand of deep learning pipelines have grownfast in recent years and thus a variety of pruning techniques have beendeveloped to reduce model parameters. The majority of these techniques focus onreducing inference costs by pruning the network after a pass of full training.A smaller number of methods address the reduction of training costs, mostlybased on compressing the network via low-rank layer factorizations. Despitetheir efficiency for linear layers, these methods fail to effectively handleconvolutional filters. In this work, we propose a low-parametric trainingmethod that factorizes the convolutions into tensor Tucker format andadaptively prunes the Tucker ranks of the convolutional kernel during training.Leveraging fundamental results from geometric integration theory ofdifferential equations on tensor manifolds, we obtain a robust trainingalgorithm that provably approximates the full baseline performance andguarantees loss descent. A variety of experiments against the full model andalternative low-rank baselines are implemented, showing that the proposedmethod drastically reduces the training costs, while achieving highperformance, comparable to or better than the full baseline, and consistentlyoutperforms competing low-rank approaches.

Rank-adaptive spectral pruning of convolutional layers during training