English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the past few years, neuroimaging has entered the Big Data era due to thejoint increase in image resolution, data sharing, and study sizes. However, noparticular Big Data engines have emerged in this field, and severalalternatives remain available. We compare two popular Big Data engines withPython APIs, Apache Spark and Dask, for their runtime performance in processingneuroimaging pipelines. Our evaluation uses two synthetic pipelines processingthe 81GB BigBrain image, and a real pipeline processing anatomical data frommore than 1,000 subjects. We benchmark these pipelines using variouscombinations of task durations, data sizes, and numbers of workers, deployed onan 8-node (8 cores ea.) compute cluster in Compute Canada's Arbutus cloud. Weevaluate PySpark's RDD API against Dask's Bag, Delayed and Futures. Resultsshow that despite slight differences between Spark and Dask, both enginesperform comparably. However, Dask pipelines risk being limited by Python's GILdepending on task type and cluster configuration. In all cases, the majorlimiting factor was data transfer. While either engine is suitable forneuroimaging pipelines, more effort needs to be placed in reducing datatransfer time.

A performance comparison of Dask and Apache Spark for data-intensive neuroimaging pipelines