English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider offline reinforcement learning (RL) with heterogeneous agentsunder severe data scarcity, i.e., we only observe a single historicaltrajectory for every agent under an unknown, potentially sub-optimal policy. Wefind that the performance of state-of-the-art offline and model-based RLmethods degrade significantly given such limited data availability, even forcommonly perceived "solved" benchmark settings such as "MountainCar" and"CartPole". To address this challenge, we propose PerSim, a model-based offlineRL approach which first learns a personalized simulator for each agent bycollectively using the historical trajectories across all agents, prior tolearning a policy. We do so by positing that the transition dynamics acrossagents can be represented as a latent function of latent factors associatedwith agents, states, and actions; subsequently, we theoretically establish thatthis function is well-approximated by a "low-rank" decomposition of separableagent, state, and action latent functions. This representation suggests asimple, regularized neural network architecture to effectively learn thetransition dynamics per agent, even with scarce, offline data. We performextensive experiments across several benchmark environments and RL methods. Theconsistent improvement of our approach, measured in terms of both statedynamics prediction and eventual reward, confirms the efficacy of our frameworkin leveraging limited historical data to simultaneously learn personalizedpolicies across agents.

PerSim: Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents via Personalized Simulators