English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Structured pruning is a commonly used technique in deploying deep neuralnetworks (DNNs) onto resource-constrained devices. However, the existingpruning methods are usually heuristic, task-specified, and require an extrafine-tuning procedure. To overcome these limitations, we propose a frameworkthat compresses DNNs into slimmer architectures with competitive performancesand significant FLOPs reductions by Only-Train-Once (OTO). OTO contains twokeys: (i) we partition the parameters of DNNs into zero-invariant groups,enabling us to prune zero groups without affecting the output; and (ii) topromote zero groups, we then formulate a structured-sparsity optimizationproblem and propose a novel optimization algorithm, Half-Space StochasticProjected Gradient (HSPG), to solve it, which outperforms the standard proximalmethods on group sparsity exploration and maintains comparable convergence. Todemonstrate the effectiveness of OTO, we train and compress full modelssimultaneously from scratch without fine-tuning for inference speedup andparameter reduction, and achieve state-of-the-art results on VGG16 for CIFAR10,ResNet50 for CIFAR10 and Bert for SQuAD and competitive result on ResNet50 forImageNet. The source code is available athttps://github.com/tianyic/only_train_once.

Only Train Once: A One-Shot Neural Network Training And Pruning Framework