English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

With the rapid development of artificial general intelligence (AGI), variousmultimedia services based on pretrained foundation models (PFMs) need to beeffectively deployed. With edge servers that have cloud-level computing power,edge intelligence can extend the capabilities of AGI to mobile edge networks.However, compared with cloud data centers, resource-limited edge servers canonly cache and execute a small number of PFMs, which typically consist ofbillions of parameters and require intensive computing power and GPU memoryduring inference. To address this challenge, in this paper, we propose a jointfoundation model caching and inference framework that aims to balance thetradeoff among inference latency, accuracy, and resource consumption bymanaging cached PFMs and user requests efficiently during the provisioning ofgenerative AI services. Specifically, considering the in-context learningability of PFMs, a new metric named the Age of Context (AoC), is proposed tomodel the freshness and relevance between examples in past demonstrations andcurrent service requests. Based on the AoC, we propose a least context cachingalgorithm to manage cached PFMs at edge servers with historical prompts andinference results. The numerical results demonstrate that the proposedalgorithm can reduce system costs compared with existing baselines byeffectively utilizing contextual information.

Joint Foundation Model Caching and Inference of Generative AI Services for Edge Intelligence