English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Recently, emergence has received widespread attention from the researchcommunity along with the success of large-scale models. Different from theliterature, we hypothesize a key factor that promotes the performance duringthe increase of scale: the reduction of monosemantic neurons that can only formone-to-one correlations with specific features. Monosemantic neurons tend to besparser and have negative impacts on the performance in large models. Inspiredby this insight, we propose an intuitive idea to identify monosemantic neuronsand inhibit them. However, achieving this goal is a non-trivial task as thereis no unified quantitative evaluation metric and simply banning monosemanticneurons does not promote polysemanticity in neural networks. Therefore, wefirst propose a new metric to measure the monosemanticity of neurons with theguarantee of efficiency for online computation, then introduce a theoreticallysupported method to suppress monosemantic neurons and proactively promote theratios of polysemantic neurons in training neural networks. We validate ourconjecture that monosemanticity brings about performance change at differentmodel scales on a variety of neural networks and benchmark datasets indifferent areas, including language, image, and physics simulation tasks.Further experiments validate our analysis and theory regarding the inhibitionof monosemanticity.

Learning from Emergence: A Study on Proactively Inhibiting the Monosemantic Neurons of Artificial Neural Networks