English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The wide adoption of DNNs has given birth to unrelenting computingrequirements, forcing datacenter operators to adopt domain-specificaccelerators to train them. These accelerators typically employ densely packedfull precision floating-point arithmetic to maximize performance per area.Ongoing research efforts seek to further increase that performance density byreplacing floating-point with fixed-point arithmetic. However, a significantroadblock for these attempts has been fixed point's narrow dynamic range, whichis insufficient for DNN training convergence. We identify block floating point(BFP) as a promising alternative representation since it exhibits wide dynamicrange and enables the majority of DNN operations to be performed withfixed-point logic. Unfortunately, BFP alone introduces several limitations thatpreclude its direct applicability. In this work, we introduce HBFP, a hybridBFP-FP approach, which performs all dot products in BFP and other operations infloating point. HBFP delivers the best of both worlds: the high accuracy offloating point at the superior hardware density of fixed point. For a widevariety of models, we show that HBFP matches floating point's accuracy whileenabling hardware implementations that deliver up to 8.5x higher throughput.

Training DNNs with Hybrid Block Floating Point