English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Training neural networks with discrete stochastic variables presents a uniquechallenge. Backpropagation is not directly applicable, nor are thereparameterization tricks used in networks with continuous stochasticvariables. To address this challenge, we present Hindsight Network CreditAssignment (HNCA), a novel gradient estimation algorithm for networks ofdiscrete stochastic units. HNCA works by assigning credit to each unit based onthe degree to which its output influences its immediate children in thenetwork. We prove that HNCA produces unbiased gradient estimates with reducedvariance compared to the REINFORCE estimator, while the computational cost issimilar to that of backpropagation. We first apply HNCA in a contextual banditsetting to optimize a reward function that is unknown to the agent. In thissetting, we empirically demonstrate that HNCA significantly outperformsREINFORCE, indicating that the variance reduction implied by our theoreticalanalysis is significant and impactful. We then show how HNCA can be extended tooptimize a more general function of the outputs of a network of stochasticunits, where the function is known to the agent. We apply this extended versionof HNCA to train a discrete variational auto-encoder and empirically show itcompares favourably to other strong methods. We believe that the ideasunderlying HNCA can help stimulate new ways of thinking about efficient creditassignment in stochastic compute graphs.

Hindsight Network Credit Assignment: Efficient Credit Assignment in Networks of Discrete Stochastic Units