English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Residual networks have shown great success and become indispensable intoday's deep models. In this work, we aim to re-investigate the trainingprocess of residual networks from a novel social psychology perspective ofloafing, and further propose a new training strategy to strengthen theperformance of residual networks. As residual networks can be viewed asensembles of relatively shallow networks (i.e., \textit{unraveled view}) inprior works, we also start from such view and consider that the finalperformance of a residual network is co-determined by a group of sub-networks.Inspired by the social loafing problem of social psychology, we find thatresidual networks invariably suffer from similar problem, where sub-networks ina residual network are prone to exert less effort when working as part of thegroup compared to working alone. We define this previously overlooked problemas \textit{network loafing}. As social loafing will ultimately cause the lowindividual productivity and the reduced overall performance, network loafingwill also hinder the performance of a given residual network and itssub-networks. Referring to the solutions of social psychology, we propose\textit{stimulative training}, which randomly samples a residual sub-networkand calculates the KL-divergence loss between the sampled sub-network and thegiven residual network, to act as extra supervision for sub-networks and makethe overall goal consistent. Comprehensive empirical results and theoreticalanalyses verify that stimulative training can well handle the loafing problem,and improve the performance of a residual network by improving the performanceof its sub-networks. The code is available athttps://github.com/Sunshine-Ye/NIPS22-ST .

Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing