English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In recommender systems (RecSys) and real-time bidding (RTB) for onlineadvertisements, we often try to optimize sequential decision making usingbandit and reinforcement learning (RL) techniques. In these applications,offline reinforcement learning (offline RL) and off-policy evaluation (OPE) arebeneficial because they enable safe policy optimization using only logged datawithout any risky online interaction. In this position paper, we explore thepotential of using simulation to accelerate practical research of offline RLand OPE, particularly in RecSys and RTB. Specifically, we discuss howsimulation can help us conduct empirical research of offline RL and OPE. Wetake a position to argue that we should effectively use simulations in theempirical research of offline RL and OPE. To refute the counterclaim thatexperiments using only real-world data are preferable, we first point out theunderlying risks and reproducibility issue in real-world experiments. Then, wedescribe how these issues can be addressed by using simulations. Moreover, weshow how to incorporate the benefits of both real-world and simulation-basedexperiments to defend our position. Finally, we also present an open challengeto further facilitate practical research of offline RL and OPE in RecSys andRTB, with respect to public simulation platforms. As a possible solution forthe issue, we show our ongoing open source project and its potential use case.We believe that building and utilizing simulation-based evaluation platformsfor offline RL and OPE will be of great interest and relevance for the RecSysand RTB community.

Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation