English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Top-down connections in the biological brain has been shown to be importantin high cognitive functions. However, the function of this mechanism in machinelearning has not been defined clearly. In this study, we propose to lay out aframework constituted by a bottom-up and a top-down network. Here, we use aTop-down Credit Assignment Network (TDCA-network) to replace the loss functionand back propagation (BP) which serve as the feedback mechanism in traditionalbottom-up network training paradigm. Our results show that the credit given bywell-trained TDCA-network outperforms the gradient from backpropagation inclassification task under different settings on multiple datasets. In addition,we successfully use a credit diffusing trick, which can keep training andtesting performance remain unchanged, to reduce parameter complexity of theTDCA-network. More importantly, by comparing their trajectories in theparameter landscape, we find that TDCA-network directly achieved a globaloptimum, in contrast to that backpropagation only can gain a localized optimum.Thus, our results demonstrate that TDCA-network not only provide a biologicalplausible learning mechanism, but also has the potential to directly achieveglobal optimum, indicating that top-down credit assignment can substitutebackpropagation, and provide a better learning framework for Deep NeuralNetworks.

Replacing Backpropagation with Biological Plausible Top-down Credit Assignment in Deep Neural Networks Training