English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Big data processing at the production scale presents a highly complexenvironment for resource optimization (RO), a problem crucial for meetingperformance goals and budgetary constraints of analytical users. The RO problemis challenging because it involves a set of decisions (the partition count,placement of parallel instances on machines, and resource allocation to eachinstance), requires multi-objective optimization (MOO), and is compounded bythe scale and complexity of big data systems while having to meet stringenttime constraints for scheduling. This paper presents a MaxCompute-basedintegrated system to support multi-objective resource optimization viafine-grained instance-level modeling and optimization. We propose a newarchitecture that breaks RO into a series of simpler problems, new fine-grainedpredictive models, and novel optimization methods that exploit these models tomake effective instance-level recommendations in a hierarchical MOO framework.Evaluation using production workloads shows that our new RO system could reduce37-72% latency and 43-78% cost at the same time, compared to the currentoptimizer and scheduler, while running in 0.02-0.23s.

Fine-Grained Modeling and Optimization for Intelligent Resource Management in Big Data Processing