English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

On-device machine learning (ML) inference can enable the use of private userdata on user devices without revealing them to remote servers. However, a pureon-device solution to private ML inference is impractical for many applicationsthat rely on embedding tables that are too large to be stored on-device. Inparticular, recommendation models typically use multiple embedding tables eachon the order of 1-10 GBs of data, making them impractical to store on-device.To overcome this barrier, we propose the use of private information retrieval(PIR) to efficiently and privately retrieve embeddings from servers withoutsharing any private information. As off-the-shelf PIR algorithms are usuallytoo computationally intensive to directly use for latency-sensitive inferencetasks, we 1) propose novel GPU-based acceleration of PIR, and 2) co-design PIRwith the downstream ML application to obtain further speedup. Our GPUacceleration strategy improves system throughput by more than $20 \times$ overan optimized CPU PIR implementation, and our PIR-ML co-design provides an over$5 \times$ additional throughput improvement at fixed model quality. Together,for various on-device ML applications such as recommendation and languagemodeling, our system on a single V100 GPU can serve up to $100,000$ queries persecond -- a $>100 \times$ throughput improvement over a CPU-based baseline --while maintaining model accuracy.

GPU-based Private Information Retrieval for On-Device Machine Learning Inference