English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reducing the latency variance in machine learning inference is a keyrequirement in many applications. Variance is harder to control in a clouddeployment in the presence of stragglers. In spite of this challenge, inferenceis increasingly being done in the cloud, due to the advent of affordablemachine learning as a service (MLaaS) platforms. Existing approaches to reducevariance rely on replication which is expensive and partially negates theaffordability of MLaaS. In this work, we argue that MLaaS platforms alsoprovide unique opportunities to cut the cost of redundancy. In MLaaS platforms,multiple inference requests are concurrently received by a load balancer whichcan then create a more cost-efficient redundancy coding across a largercollection of images. We propose a novel convolutional neural network model,Collage-CNN, to provide a low-cost redundancy framework. A Collage-CNN modeltakes a collage formed by combining multiple images and performs multi-imageclassification in one shot, albeit at slightly lower accuracy. We then augmenta collection of traditional single image classifiers with a single Collage-CNNclassifier which acts as a low-cost redundant backup. Collage-CNN then providesbackup classification results if a single image classification straggles.Deploying the Collage-CNN models in the cloud, we demonstrate that the 99thpercentile tail latency of inference can be reduced by 1.47X compared toreplication based approaches while providing high accuracy. Also, variation ininference latency can be reduced by 9X with a slight increase in averageinference latency.

Collage Inference: Achieving low tail latency during distributed image classification using coded redundancy models