English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Out-of-distribution (OOD) detection is a well-studied topic in supervisedlearning. Extending the successes in supervised learning methods to thereinforcement learning (RL) setting, however, is difficult due to the datagenerating process - RL agents actively query their environment for data, andthe data are a function of the policy followed by the agent. An agent couldthus neglect a shift in the environment if its policy did not lead it toexplore the aspect of the environment that shifted. Therefore, to achieve safeand robust generalization in RL, there exists an unmet need for OOD detectionthrough active experimentation. Here, we attempt to bridge this lacuna by firstdefining a causal framework for OOD scenarios or environments encountered by RLagents in the wild. Then, we propose a novel task: that of Out-of-TaskDistribution (OOTD) detection. We introduce an RL agent that activelyexperiments in a test environment and subsequently concludes whether it is OOTDor not. We name our method GalilAI, in honor of Galileo Galilei, as itdiscovers, among other causal processes, that gravitational acceleration isindependent of the mass of a body. Finally, we propose a simple probabilisticneural network baseline for comparison, which extends extant Model-Based RL. Wefind that GalilAI outperforms the baseline significantly. See visualizations ofour method https://galil-ai.github.io/

GalilAI: Out-of-Task Distribution Detection using Causal Active Experimentation for Safe Transfer RL