English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Recent advances in deep learning optimization showed that, with somea-posteriori information on fully-trained models, it is possible to match thesame performance by simply training a subset of their parameters. Such adiscovery has a broad impact from theory to applications, driving the researchtowards methods to identify the minimum subset of parameters to train withoutlook-ahead information exploitation. However, the methods proposed do not matchthe state-of-the-art performance, and rely on unstructured sparsely connectedmodels. In this work we shift our focus from the single parameters to thebehavior of the whole neuron, exploiting the concept of neuronal equilibrium(NEq). When a neuron is in a configuration at equilibrium (meaning that it haslearned a specific input-output relationship), we can halt its update; on thecontrary, when a neuron is at non-equilibrium, we let its state evolve towardsan equilibrium state, updating its parameters. The proposed approach has beentested on different state-of-the-art learning strategies and tasks, validatingNEq and observing that the neuronal equilibrium depends on the specificlearning setup.

To update or not to update? Neurons at equilibrium in deep models