English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently there has been significant interest in training machine-learningmodels at low precision: by reducing precision, one can reduce computation andcommunication by one order of magnitude. We examine training at reducedprecision, both from a theoretical and practical perspective, and ask: is itpossible to train models at end-to-end low precision with provable guarantees?Can this lead to consistent order-of-magnitude speedups? We present a frameworkcalled ZipML to answer these questions. For linear models, the answer is yes.We develop a simple framework based on one simple but novel strategy calleddouble sampling. Our framework is able to execute training at low precisionwith no bias, guaranteeing convergence, whereas naive quantization wouldintroduce significant bias. We validate our framework across a range ofapplications, and show that it enables an FPGA prototype that is up to 6.5xfaster than an implementation using full 32-bit precision. We further develop avariance-optimal stochastic quantization strategy and show that it can make asignificant difference in a variety of settings. When applied to linear modelstogether with double sampling, we save up to another 1.7x in data movementcompared with uniform quantization. When training deep networks with quantizedmodels, we achieve higher accuracy than the state-of-the-art XNOR-Net. Finally,we extend our framework through approximation to non-linear models, such asSVM. We show that, although using low-precision data induces bias, we canappropriately bound and control the bias. We find in practice 8-bit precisionis often sufficient to converge to the correct solution. Interestingly,however, in practice we notice that our framework does not always outperformthe naive rounding approach. We discuss this negative result in detail.

The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning