English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Due to the surge of cloud-assisted AI services, the problem of designingresilient prediction serving systems that can effectively cope withstragglers/failures and minimize response delays has attracted much interest.The common approach for tackling this problem is replication which assigns thesame prediction task to multiple workers. This approach, however, is veryinefficient and incurs significant resource overheads. Hence, a learning-basedapproach known as parity model (ParM) has been recently proposed which learnsmodels that can generate parities for a group of predictions in order toreconstruct the predictions of the slow/failed workers. While thislearning-based approach is more resource-efficient than replication, it istailored to the specific model hosted by the cloud and is particularly suitablefor a small number of queries (typically less than four) and tolerating veryfew (mostly one) number of stragglers. Moreover, ParM does not handle Byzantineadversarial workers. We propose a different approach, named Approximate CodedInference (ApproxIFER), that does not require training of any parity models,hence it is agnostic to the model hosted by the cloud and can be readilyapplied to different data domains and model architectures. Compared withearlier works, ApproxIFER can handle a general number of stragglers and scalessignificantly better with the number of queries. Furthermore, ApproxIFER isrobust against Byzantine workers. Our extensive experiments on a large numberof datasets and model architectures also show significant accuracy improvementby up to 58% over the parity model approaches.

ApproxIFER: A Model-Agnostic Approach to Resilient and Robust Prediction Serving Systems