English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deploying Large Language Models (LLMs) locally on mobile devices presents asignificant challenge due to their extensive memory requirements. In thispaper, we introduce LinguaLinked, a system for decentralized, distributed LLMinference on mobile devices. LinguaLinked enables collaborative execution ofthe inference task across multiple trusted devices. LinguaLinked ensures dataprivacy by processing information locally. LinguaLinked uses three keystrategies. First, an optimized model assignment technique segments LLMs anduses linear optimization to align segments with each device's capabilities.Second, an optimized data transmission mechanism ensures efficient andstructured data flow between model segments while also maintaining theintegrity of the original model structure. Finally, LinguaLinked incorporates aruntime load balancer that actively monitors and redistributes tasks amongmobile devices to prevent bottlenecks, enhancing the system's overallefficiency and responsiveness. We demonstrate that LinguaLinked facilitatesefficient LLM inference while maintaining consistent throughput and minimallatency through extensive testing across various mobile devices, from high-endto low-end Android devices. In our evaluations, compared to the baseline,LinguaLinked achieves an inference performance acceleration of $1.11\times$ to$1.61\times$ in single-threaded settings, $1.73\times$ to $2.65\times$ withmulti-threading. Additionally, runtime load balancing yields an overallinference acceleration of $1.29\times$ to $1.32\times$.

LinguaLinked: A Distributed Large Language Model Inference System for Mobile Devices