English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study contextual bandits in the presence of a stage-wise constraint (aconstraint at each round), when the constraint must be satisfied both with highprobability and in expectation. Obviously the setting where the constraint isin expectation is a relaxation of the one with high probability. We start withthe linear case where both the contextual bandit problem (reward function) andthe stage-wise constraint (cost function) are linear. In each of the highprobability and in expectation settings, we propose an upper-confidence boundalgorithm for the problem and prove a $T$-round regret bound for it. Ouralgorithms balance exploration and constraint satisfaction using a novel ideathat scales the radii of the reward and cost confidence sets with differentscaling factors. We also prove a lower-bound for this constrained problem, showhow our algorithms and analyses can be extended to multiple constraints, andprovide simulations to validate our theoretical results. In the highprobability setting, we describe the minimum requirements for the action set inorder for our algorithm to be tractable. In the setting that the constraint isin expectation, we further specialize our results to multi-armed bandits andpropose a computationally efficient algorithm for this setting with regretanalysis. Finally, we extend our results to the case where the reward and costfunctions are both non-linear. We propose an algorithm for this case and provea regret bound for it that characterize the function class complexity by theeluder dimension.

Contextual Bandits with Stage-wise Constraints