English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The dynamic request patterns of machine learning (ML) inference workloadshave driven an increasing trend towards exploiting serverless computing forscalable ML model serving. However, today's serverless platforms lack efficientsupport for GPUs -- provisioning functions on GPUs incurs extremely highoverhead, forcing them to keep long-running even when idling for reduced coldstarts. This leads to significant resource waste to perform ML inference andhinders the pay-per-use billing for GPUs.  In this paper, we present FaaSwap, a serverless platform enablingfine-grained, request-level GPU sharing for resource-efficient ML inference.FaaSwap leverages model swapping to support fast inference execution at lowresource cost. It keeps models in a host which has a large amount of cheapmemory and quickly swaps models to GPUs when requested, reducing per-functionkeep-alive cost and enabling efficient GPU sharing across much more functions.FaaSwap also supports swapping models between GPUs for load balancing andimproved inference performance. In FaaSwap, we design sophisticated requestscheduling and memory management algorithms that efficiently exploit modelswapping to reduce GPU cost and meet latency service-level objectives (SLOs)for all inference functions. We have implemented and integrated FaaSwap intoAlibaba Cloud Function Compute (FC), one of the world's largest commercialserverless platform. Evaluation results show that FaaSwap can achievelow-latency model swapping, efficiently share a GPU across hundreds offunctions, and satisfy per-function latency SLOs at scale.

FaaSwap: SLO-Aware, GPU-Efficient Serverless Inference via Model Swapping