English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Many modern applications require real-time processing of large volumes ofhigh-speed data. Such data processing needs can be modeled as a streamingcomputation. A streaming computation is specified as a dataflow graph thatexposes multiple opportunities for parallelizing its execution, in the form ofdata, pipeline and task parallelism. On the other hand, many importantapplications require that processing of the stream be ordered, where inputs areprocessed in the same order as they arrive. There is a fundamental conflictbetween ordered processing and parallelizing the streaming computation. Thispaper focuses on the problem of effectively parallelizing ordered streamingcomputations on a shared-memory multicore machine.  We first address the key challenges in exploiting data parallelism in theordered setting. We present a low-latency, non-blocking concurrent datastructure to order outputs produced by concurrent workers on an operator. Wealso propose a new approach to parallelizing partitioned stateful operatorsthat can handle load imbalance across partitions effectively and mostly avoiddelays due to ordering. We illustrate the trade-offs and effectiveness of ourconcurrent data-structures on micro-benchmarks and streaming queries from theTPCx-BB benchmark. We then present an adaptive runtime that dynamically mapsthe exposed parallelism in the computation to that of the machine. We proposeseveral intuitive scheduling heuristics and compare them empirically on theTPCx-BB queries. We find that for streaming computations, heuristics thatexploit as much pipeline parallelism as possible perform better than those thatseek to exploit data parallelism.

Scaling Ordered Stream Processing on Shared-Memory Multicores