English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper investigates the intriguing question of whether we can createlearning algorithms that automatically generate training data, learningenvironments, and curricula in order to help AI agents rapidly learn. We showthat such algorithms are possible via Generative Teaching Networks (GTNs), ageneral approach that is, in theory, applicable to supervised, unsupervised,and reinforcement learning, although our experiments only focus on thesupervised case. GTNs are deep neural networks that generate data and/ortraining environments that a learner (e.g. a freshly initialized neuralnetwork) trains on for a few SGD steps before being tested on a target task. Wethen differentiate through the entire learning process via meta-gradients toupdate the GTN parameters to improve performance on the target task. GTNs havethe beneficial property that they can theoretically generate any type of dataor training environment, making their potential impact large. This paperintroduces GTNs, discusses their potential, and showcases that they cansubstantially accelerate learning. We also demonstrate a practical and excitingapplication of GTNs: accelerating the evaluation of candidate architectures forneural architecture search (NAS), which is rate-limited by such evaluations,enabling massive speed-ups in NAS. GTN-NAS improves the NAS state of the art,finding higher performing architectures when controlling for the searchproposal mechanism. GTN-NAS also is competitive with the overall state of theart approaches, which achieve top performance while using orders of magnitudeless computation than typical NAS methods. Speculating forward, GTNs mayrepresent a first step toward the ambitious goal of algorithms that generatetheir own training data and, in doing so, open a variety of interesting newresearch questions and directions.

Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data