English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper we propose and study a technique to reduce the number ofparameters and computation time in fully-connected layers of neural networksusing Kronecker product, at a mild cost of the prediction quality. Thetechnique proceeds by replacing Fully-Connected layers with so-called KroneckerFully-Connected layers, where the weight matrices of the FC layers areapproximated by linear combinations of multiple Kronecker products of smallermatrices. In particular, given a model trained on SVHN dataset, we are able toconstruct a new KFC model with 73\% reduction in total number of parameters,while the error only rises mildly. In contrast, using low-rank method can onlyachieve 35\% reduction in total number of parameters given similar qualitydegradation allowance. If we only compare the KFC layer with its counterpartfully-connected layer, the reduction in the number of parameters exceeds 99\%.The amount of computation is also reduced as we replace matrix product of thelarge matrices in FC layers with matrix products of a few smaller matrices inKFC layers. Further experiments on MNIST, SVHN and some Chinese Characterrecognition models also demonstrate effectiveness of our technique.

Compression of Fully-Connected Layer in Neural Network by Kronecker Product