English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Top-K queries are an established heuristic in information retrieval. Thispaper presents an approach for optimal tiered storage allocation under streamprocessing workloads using this heuristic: those requiring the analysis of onlythe top-$K$ ranked most relevant, or most interesting, documents from afixed-length stream, stream window, or batch job. In this workflow, documentsare analyzed relevance with a user-specified interestingness function, on whichthey are ranked, the top-$K$ being selected (and hence stored) for furtherprocessing. This workflow allows human in the loop systems, includingsupervised machine learning, to prioritize documents. This scenario bearssimilarity to the classic Secretary Hiring Problem (SHP), and the expected rateof document writes, and document lifetime, can be modelled as a function ofdocument index. We present parameter-based algorithms for storage tierplacement, minimizing document storage and transport costs. We show thatoptimal parameter values are a function of these costs. It is possible to modelapplication IO behaviour a priori for this class of workloads. When combinedwith tiered storage, the tractability of the probabilistic model of IO makes iteasy to optimize tier allocation, yielding simple analytic solutions. Thiscontrasts with (often complex) existing work on tiered storage optimization,which is either tightly coupled to specific use cases, or requires activemonitoring of application IO load (a reactive approach) -- ill-suited tolong-running or one-off operations common in the scientific computing domain.Wevalidate our model with a trace-driven simulation of a bio-chemical modelexploration, and give worked examples for two cloud storage case studies.

Adapting The Secretary Hiring Problem for Optimal Hot-Cold Tier Placement under Top-$K$ Workloads