English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Motivation: Storage of genomic data is a major cost for the Life Sciences,effectively addressed mostly via specialized data compression methods. For thesame reasons of abundance in data production, the use of Big Data technologiesis seen as the future for genomic data storage and processing, withMapReduce-Hadoop as leaders. Somewhat surprisingly, none of the specializedFASTA/Q compressors is available within Hadoop. Indeed, their deployment thereis not exactly immediate. Such a State of the Art is problematic. Results: Weprovide major advances in two different directions. Methodologically, wepropose two general methods, with the corresponding software, that make veryeasy to deploy a specialized FASTA/Q compressor within MapReduce-Hadoop forprocessing files stored on the distributed Hadoop File System, with very littleknowledge of Hadoop. Practically, we provide evidence that the deployment ofthose specialized compressors within Hadoop, not available so far, results inmajor cost savings, i.e., on large plant genomes, 30% less HDFS data blocks(one block=128MB), speed-up of at least x1.5 in I/O time and comparable orreduced network communication time with respect to the use of genericcompressors available in Hadoop. Finally, we observe that these results holdalso for the Apache Spark framework, when used to process FASTA/Q files storedon the Hadoop File System.

FASTA/Q Data Compressors for MapReduce-Hadoop Genomics:Space and Time Savings Made Easy -- Version 1