English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Fault-tolerance techniques for stream processing engines can be categorizedinto passive and active approaches. A typical passive approach periodicallycheckpoints a processing task's runtime states and can recover a failed task byrestoring its runtime state using its latest checkpoint. On the other hand, anactive approach usually employs backup nodes to run replicated tasks. Uponfailure, the active replica can take over the processing of the failed taskwith minimal latency. However, both approaches have their own inadequacies inMassively Parallel Stream Processing Engines (MPSPE). The passive approachincurs a long recovery latency especially when a number of correlated nodesfail simultaneously, while the active approach requires extra replicationresources. In this paper, we propose a new fault-tolerance framework, which isPassive and Partially Active (PPA). In a PPA scheme, the passive approach isapplied to all tasks while only a selected set of tasks will be activelyreplicated. The number of actively replicated tasks depends on the availableresources. If tasks without active replicas fail, tentative outputs will begenerated before the completion of the recovery process. We also proposeeffective and efficient algorithms to optimize a partially active replicationplan to maximize the quality of tentative outputs. We implemented PPA on top ofStorm, an open-source MPSPE and conducted extensive experiments using both realand synthetic datasets to verify the effectiveness of our approach.

Tolerating Correlated Failures in Massively Parallel Stream Processing Engines