English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Over the last decade, deep neural networks have transformed our society, andthey are already widely applied in various machine learning applications.State-of-art deep neural networks are becoming larger in size every year todeliver increasing model accuracy, and as a result, model training consumessubstantial computing resources and will only consume more in the future. Usingcurrent training methods, in each iteration, to process a data point $x \in\mathbb{R}^d$ in a layer, we need to spend $\Theta(md)$ time to evaluate allthe $m$ neurons in the layer. This means processing the entire layer takes$\Theta(nmd)$ time for $n$ data points. Recent work [Song, Yang and Zhang,NeurIPS 2021] reduces this time per iteration to $o(nmd)$, but requiresexponential time to preprocess either the data or the neural network weights,making it unlikely to have practical usage.  In this work, we present a new preprocessing method that simply stores theweight-data correlation in a tree data structure in order to quickly,dynamically detect which neurons fire at each iteration. Our method requiresonly $O(nmd)$ time in preprocessing and still achieves $o(nmd)$ time periteration. We complement our new algorithm with a lower bound, proving thatassuming a popular conjecture from complexity theory, one could notsubstantially speed up our algorithm for dynamic detection of firing neurons.

Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing