English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Backpropagation (BP) is the cornerstone of today's deep learning algorithms,but it is inefficient partially because of backward locking, which meansupdating the weights of one layer locks the weight updates in the other layers.Consequently, it is challenging to apply parallel computing or a pipelinestructure to update the weights in different layers simultaneously. In thispaper, we introduce a novel learning structure called associated learning (AL),which modularizes the network into smaller components, each of which has alocal objective. Because the objectives are mutually independent, AL can learnthe parameters in different layers independently and simultaneously, so it isfeasible to apply a pipeline structure to improve the training throughput.Specifically, this pipeline structure improves the complexity of the trainingtime from O(nl), which is the time complexity when using BP and stochasticgradient descent (SGD) for training, to O(n + l), where n is the number oftraining instances and l is the number of hidden layers. Surprisingly, eventhough most of the parameters in AL do not directly interact with the targetvariable, training deep models by this method yields accuracies comparable tothose from models trained using typical BP methods, in which all parameters areused to predict the target variable. Consequently, because of the scalabilityand the predictive power demonstrated in the experiments, AL deserves furtherstudy to determine the better hyperparameter settings, such as activationfunction selection, learning rate scheduling, and weight initialization, toaccumulate experience, as we have done over the years with the typical BPmethod. Additionally, perhaps our design can also inspire new network designsfor deep learning. Our implementation is available athttps://github.com/SamYWK/Associated_Learning.

Associated Learning: Decomposing End-to-end Backpropagation based on Auto-encoders and Target Propagation