English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Training deep neural networks (DNNs) is time-consuming. While most existingsolutions try to overlap/schedule computation and communication for efficienttraining, this paper goes one step further by skipping computing andcommunication through DNN layer freezing. Our key insight is that the trainingprogress of internal DNN layers differs significantly, and front layers oftenbecome well-trained much earlier than deep layers. To explore this, we firstintroduce the notion of training plasticity to quantify the training progressof internal DNN layers. Then we design Egeria, a knowledge-guided DNN trainingsystem that employs semantic knowledge from a reference model to accuratelyevaluate individual layers' training plasticity and safely freeze the convergedones, saving their corresponding backward computation and communication. Ourreference model is generated on the fly using quantization techniques and runsforward operations asynchronously on available CPUs to minimize the overhead.In addition, Egeria caches the intermediate outputs of the frozen layers withprefetching to further skip the forward computation. Our implementation andtestbed experiments with popular vision and language models show that Egeriaachieves 19%-43% training speedup w.r.t. the state-of-the-art withoutsacrificing accuracy.

Egeria: Efficient DNN Training with Knowledge-Guided Layer Freezing