English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Tensor decompositions have been successfully applied to compress neuralnetworks. The compression algorithms using tensor decompositions commonlyminimize the approximation error on the weights. Recent work assumes theapproximation error on the weights is a proxy for the performance of the modelto compress multiple layers and fine-tune the compressed model. Surprisingly,little research has systematically evaluated which approximation errors can beused to make choices regarding the layer, tensor decomposition method, andlevel of compression. To close this gap, we perform an experimental study totest if this assumption holds across different layers and types ofdecompositions, and what the effect of fine-tuning is. We include theapproximation error on the features resulting from a compressed layer in ouranalysis to test if this provides a better proxy, as it explicitly takes thedata into account. We find the approximation error on the weights has apositive correlation with the performance error, before as well as afterfine-tuning. Basing the approximation error on the features does not improvethe correlation significantly. While scaling the approximation error commonlyis used to account for the different sizes of layers, the average correlationacross layers is smaller than across all choices (i.e. layers, decompositions,and level of compression) before fine-tuning. When calculating the correlationacross the different decompositions, the average rank correlation is largerthan across all choices. This means multiple decompositions can be consideredfor compression and the approximation error can be used to choose between them.

How Informative is the Approximation Error from Tensor Decomposition for Neural Network Compression?