English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Inspired by the cache replacement problem, we propose and solve a new variantof the well-known multi-armed bandit (MAB), thus providing a solution forimproving existing state-of-the-art cache management methods. Each arm (orexpert) represents a distinct cache replacement policy, which advises on thepage to evict from the cache when needed. Feedback on the eviction comes in theform of a "miss", but at an indeterminate time after the action is taken, andthe cost of the eviction is set to be inversely proportional to the responsetime. The feedback is ignored if it comes after a threshold value for thedelay, which we set to be equal to the size of the page eviction history. Thus,for delays beyond the threshold, its cost is assumed to be zero. Consequently,we call this problem with delayed feedback and decaying costs. We introduce anadaptive reinforcement learning algorithm EXP4-DFDC that provides a solution tothe problem. We derive an optimal learning rate for EXP4-DFDC that defines thebalance between exploration and exploitation and proves theoretically that theexpected regret of our algorithm is a vanishing quantity as a function of time.As an application, we show that LeCaR, a recent top-performing machine learningalgorithm for cache replacement, can be enhanced with adaptive learning usingour formulations. We present an improved adaptive version of LeCaR, calledOLeCaR, with the learning rate set as determined by the theoretical derivationpresented here to minimize regret for EXP4-DFDC. It then follows that LeCaR andOLeCaR are theoretically guaranteed to have vanishing regret over time.

Cache Replacement as a MAB with Delayed Feedback and Decaying Costs