English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We show how forward-mode automatic differentiation (AD) can be employedwithin larger reverse-mode computations to dynamically differentiate broadcastoperations in a GPU-friendly manner. Our technique fully exploits the broadcastJacobian's inherent sparsity structure, and unlike a pure reverse-modeapproach, this "mixed-mode" approach does not require a backwards pass over thebroadcasted operation's subgraph, obviating the need for severalreverse-mode-specific programmability restrictions on user-authored broadcastoperations. Most notably, this approach allows broadcast fusion in primal codedespite the presence of data-dependent control flow. We discuss an experimentin which a Julia implementation of our technique outperformed pure reverse-modeTensorFlow and Julia implementations for differentiating through broadcastoperations within an HM-LSTM cell update calculation.

Dynamic Automatic Differentiation of GPU Broadcast Kernels