English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many machine learning problems encode their data as a matrix with a possiblyvery large number of rows and columns. In several applications likeneuroscience, image compression or deep reinforcement learning, the principalsubspace of such a matrix provides a useful, low-dimensional representation ofindividual data. Here, we are interested in determining the $d$-dimensionalprincipal subspace of a given matrix from sample entries, i.e. from smallrandom submatrices. Although a number of sample-based methods exist for thisproblem (e.g. Oja's rule \citep{oja1982simplified}), these assume access tofull columns of the matrix or particular matrix structure such as symmetry andcannot be combined as-is with neural networks \citep{baldi1989neural}. In thispaper, we derive an algorithm that learns a principal subspace from sampleentries, can be applied when the approximate subspace is represented by aneural network, and hence can be scaled to datasets with an effectivelyinfinite number of rows and columns. Our method consists in defining a lossfunction whose minimizer is the desired principal subspace, and constructing agradient estimate of this loss whose bias can be controlled. We complement ourtheoretical analysis with a series of experiments on synthetic matrices, theMNIST dataset \citep{lecun2010mnist} and the reinforcement learning domainPuddleWorld \citep{sutton1995generalization} demonstrating the usefulness ofour approach.

A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces