English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

As numerous machine learning and other algorithms increase in complexity anddata requirements, distributed computing becomes necessary to satisfy thegrowing computational and storage demands, because it enables parallelexecution of smaller tasks that make up a large computing job. However, randomfluctuations in task service times lead to straggling tasks with long executiontimes. Redundancy, in the form of task replication and erasure coding, providesdiversity that allows a job to be completed when only a subset of redundanttasks is executed, thus removing the dependency on the straggling tasks. Insituations of constrained resources (here a fixed number of parallel servers),increasing redundancy reduces the available resources for parallelism. In thispaper, we characterize the diversity vs. parallelism trade-off and identify theoptimal strategy, among replication, coding and splitting, which minimizes theexpected job completion time. We consider three common service timedistributions and establish three models that describe scaling of thesedistributions with the task size. We find that different distributions withdifferent scaling models operate optimally at different levels of redundancy,and thus may require very different code rates.

Diversity/Parallelism Trade-off in Distributed Systems with Redundancy