English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A core challenge in the interpretation of deep neural networks is identifyingcommonalities between the underlying algorithms implemented by distinctnetworks trained for the same task. Motivated by this problem, we introduceDYNAMO, an algorithm that constructs low-dimensional manifolds where each pointcorresponds to a neural network model, and two points are nearby if thecorresponding neural networks enact similar high-level computational processes.DYNAMO takes as input a collection of pre-trained neural networks and outputs ameta-model that emulates the dynamics of the hidden states as well as theoutputs of any model in the collection. The specific model to be emulated isdetermined by a model embedding vector that the meta-model takes as input;these model embedding vectors constitute a manifold corresponding to the givenpopulation of models. We apply DYNAMO to both RNNs and CNNs, and find that theresulting model embedding spaces enable novel applications: clustering ofneural networks on the basis of their high-level computational processes in amanner that is less sensitive to reparameterization; model averaging of severalneural networks trained on the same task to arrive at a new, operable neuralnetwork with similar task performance; and semi-supervised learning viaoptimization on the model embedding space. Using a fixed-point analysis ofmeta-models trained on populations of RNNs, we gain new insights into howsimilarities of the topology of RNN dynamics correspond to similarities oftheir high-level computational processes.

Analyzing Populations of Neural Networks via Dynamical Model Embedding