English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Stream processing engines enable modern systems to conduct large-scaleanalytics over unbounded data streams in real time. They often view anapplication as a direct acyclic graph with streams flowing through pipelinedinstances of various processing units. One key challenge that emerges isinstance placement, i.e., to decide the placement of instances across serverswith minimum traffic across servers and maximum resource utilization. Thechallenge roots in not only its intrinsic complexity but also the impactbetween successive application deployments. Most updated engines such as ApacheHeron exploits a more modularized scheduler design that decomposes the taskinto two stages: One decides the instance-to-container mapping while the otherfocuses on the container-to-server mapping that is delegated to standaloneresource managers. The unaligned objectives and scheduler designs in the twostages may lead to long response times or low utilization. However, so farlittle work has appeared to address the challenge. Inspired by the recentsuccess of applications of Monte Carlo Tree Search (MCTS) methods in variousfields, we develop a novel model to characterize such systems, formulate theproblem, and cast each stage of mapping into a sequential decision process. Byadopting MCTS methods, we propose MIPS, an MCTS-based Instance Placement Schemeto decide the two-staged mapping in a timely yet efficient manner. In addition,we discuss practical issues and refine MIPS to further improve its performance.Results from extensive simulations show, given mild-value of samples, MIPSoutperforms existing schemes with a significant traffic reduction andutilization improvement. To our best knowledge, this paper is the first tostudy the two-staged mapping problem and to apply MCTS to solving thechallenge.

MIPS: Instance Placement for Stream Processing Systems based on Monte Carlo Tree Search