English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The slowness principle is a concept inspired by the visual cortex of thebrain. It postulates that the underlying generative factors of a quicklyvarying sensory signal change on a slower time scale. Unsupervised learning ofintermediate representations utilizing abundant unlabeled sensory data can beleveraged to perform data-efficient supervised downstream regression. In thispaper, we propose a general formulation of slowness for unsupervisedrepresentation learning adding a slowness regularization term to the estimatelower bound of the beta-VAE to encourage temporal similarity in observation andlatent space. Within this framework we compare existing slowness regularizationterms such as the L1 and L2 loss used in existing end-to-end methods, theSlowVAE and propose a new term based on Brownian motion. We empiricallyevaluate these slowness regularization terms with respect to their downstreamtask performance and data efficiency. We find that slow representations lead toequal or better downstream task performance and data efficiency in differentexperiment domains when compared to representations without slownessregularization. Finally, we discuss how the Frechet Inception Distance (FID),traditionally used to determine the generative capabilities of GANs, can serveas a measure to predict the performance of pre-trained Autoencoder model in asupervised downstream task and accelerate hyperparameter search.

Autoencoding Slow Representations for Semi-supervised Data Efficient Regression