English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With ever-increasing volumes of scientific data produced by HPC applications,significantly reducing data size is critical because of limited capacity ofstorage space and potential bottlenecks on I/O or networks in writing/readingor transferring data. SZ and ZFP are the two leading lossy compressorsavailable to compress scientific data sets. However, their performance is notconsistent across different data sets and across different fields of some datasets: for some fields SZ provides better compression performance, while otherfields are better compressed with ZFP. This situation raises the need for anautomatic online (during compression) selection between SZ and ZFP, with aminimal overhead. In this paper, the automatic selection optimizes therate-distortion, an important statistical quality metric based on thesignal-to-noise ratio. To optimize for rate-distortion, we investigate theprinciples of SZ and ZFP. We then propose an efficient online, low-overheadselection algorithm that predicts the compression quality accurately for twocompressors in early processing stages and selects the best-fit compressor foreach data field. We implement the selection algorithm into an open-sourcelibrary, and we evaluate the effectiveness of our proposed solution againstplain SZ and ZFP in a parallel environment with 1,024 cores. Evaluation resultson three data sets representing about 100 fields show that our selectionalgorithm improves the compression ratio up to 70% with the same level of datadistortion because of very accurate selection (around 99%) of the best-fitcompressor, with little overhead (less than 7% in the experiments).

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP