English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Machine learning (ML) computations commonly execute on expensive specializedhardware, such as GPUs and TPUs, which provide high FLOPs andperformance-per-watt. For cost efficiency, it is essential to keep theseaccelerators highly utilized. This requires preprocessing input data at therate at which the accelerators can ingest and perform ML computations on thedata. To avoid data stalls, the host CPU and RAM required for input dataprocessing per accelerator core used for ML computations varies across jobs.Hence, the traditional approach of processing input data on ML acceleratorhosts with a fixed hardware ratio leads to either under-utilizing theaccelerators or the host CPU and RAM. In this paper, we address these concernsby building a disaggregated ML data processing system.  We present tf.data service, an open-source disaggregated input dataprocessing service built on top of tf.data in TensorFlow. We show thatdisaggregating data preprocessing has three key advantages for large-scale MLtraining jobs. First, the service can horizontally scale-out to right-sizeCPU/RAM host resources for data processing in each job, saving 32x trainingtime and 26x cost, on average. Second, the service can share ephemeralpreprocessed data results across jobs, to optimize CPU usage and reduceredundant computations. Finally, the service supports coordinated reads, atechnique that avoids stragglers due to different input sizes in distributedtraining, reducing training time by 2.2x, on average. Our design is inspired bylessons learned from deploying tf.data service in production, includingrelaxing data visitation guarantees without impacting model accuracy.

tf.data service: A Case for Disaggregating ML Input Data Processing