English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep Neural Network (DNN) inference is emerging as the fundamental bedrockfor a multitude of utilities and services. CPUs continue to scale up their rawcompute capabilities for DNN inference along with mature high performancelibraries to extract optimal performance. While general purpose CPUs offerunique attractive advantages for DNN inference at both datacenter and edge,they have primarily evolved to optimize single thread performance. For highlyparallel, throughput-oriented DNN inference, this results in inefficiencies inboth power and performance, impacting both raw performance scaling and overallperformance/watt.  We present Proximu$\$$, where we systematically tackle the rootinefficiencies in power and performance scaling for CPU DNN inference.Performance scales efficiently by distributing light-weight tensor compute nearall caches in a multi-level cache hierarchy. This maximizes the cumulativeutilization of the existing bandwidth resources in the system and minimizesmovement of data. Power is drastically reduced through simple ISA extensionsthat encode the structured, loop-y workload behavior. This enables a bulkoffload of pre-decoded work, with loop unrolling in the light-weight near-cacheunits, effectively bypassing the power-hungry stages of the wide Out-of-Order(OOO) CPU pipeline.  Across a number of DNN models, Proximu$\$$ achieves a 2.3x increase inconvolution performance/watt with a 2x to 3.94x scaling in raw performance.Similarly, Proximu$\$$ achieves a 1.8x increase in inner-productperformance/watt with 2.8x scaling in performance. With no changes to theprogramming model, no increase in cache capacity or bandwidth and minimaladditional hardware, Proximu$\$$ enables unprecedented CPU efficiency gainswhile achieving similar performance to state-of-the-art Domain SpecificAccelerators (DSA) for DNN inference in this AI era.

Proximu$: Efficiently Scaling DNN Inference in Multi-core CPUs through Near-Cache Compute