English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Data locality is a fundamental issue for data-parallel applications.Considering MapReduce in Hadoop, the map task scheduling part requires anefficient algorithm which takes data locality into consideration; otherwise,the system may become unstable under loads inside the system's capacity regionand jobs may experience longer completion times which are not of interest. Thedata chunk needed for any map task can be in memory, on a local disk, in alocal rack, in the same cluster or even in another data center. Hence, unlessthere has been much work on improving the speed of data center networks,different levels of service rates still exist for a task depending on where itsdata chunk is saved and from which server it receives service. Most of thetheoretical work on load balancing is for systems with two levels of datalocality including the Pandas algorithm by Xie et al. and the JSQ-MW algorithmby Wang et al., where the former is both throughput and heavy-traffic optimal,while the latter is only throughput optimal, but heavy-traffic optimal in onlya special traffic load. We show that an extension of the JSQ-MW algorithm for asystem with thee levels of data locality is throughput optimal, but notheavy-traffic optimal for all loads, only for a special traffic scenario.Furthermore, we show that the Pandas algorithm is not even throughput optimalfor a system with three levels of data locality. We then propose a novelalgorithm, Balanced-Pandas, which is both throughput and heavy-traffic optimal.To the best of our knowledge, this is the first theoretical work on loadbalancing for a system with more than two levels of data locality. This is morechallenging than two levels of data locality as a dilemma between performanceand throughput emerges.

Near-Data Scheduling for Data Centers with Multiple Levels of Data Locality