English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning representations of neural network weights given a model zoo is anemerging and challenging area with many potential applications from modelinspection, to neural architecture search or knowledge distillation. Recently,an autoencoder trained on a model zoo was able to learn a hyper-representation,which captures intrinsic and extrinsic properties of the models in the zoo. Inthis work, we extend hyper-representations for generative use to sample newmodel weights. We propose layer-wise loss normalization which we demonstrate iskey to generate high-performing models and several sampling methods based onthe topology of hyper-representations. The models generated using our methodsare diverse, performant and capable to outperform strong baselines as evaluatedon several downstream tasks: initialization, ensemble sampling and transferlearning. Our results indicate the potential of knowledge aggregation frommodel zoos to new models via hyper-representations thereby paving the avenuefor novel research directions.

Hyper-Representations as Generative Models: Sampling Unseen Neural Network Weights