English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A prominent challenge of offline reinforcement learning (RL) is the issue ofhidden confounding: unobserved variables may influence both the actions takenby the agent and the observed outcomes. Hidden confounding can compromise thevalidity of any causal conclusion drawn from data and presents a major obstacleto effective offline RL. In the present paper, we tackle the problem of hiddenconfounding in the nonidentifiable setting. We propose a definition ofuncertainty due to hidden confounding bias, termed delphic uncertainty, whichuses variation over world models compatible with the observations, anddifferentiate it from the well-known epistemic and aleatoric uncertainties. Wederive a practical method for estimating the three types of uncertainties, andconstruct a pessimistic offline RL algorithm to account for them. Our methoddoes not assume identifiability of the unobserved confounders, and attempts toreduce the amount of confounding bias. We demonstrate through extensiveexperiments and ablations the efficacy of our approach on a sepsis managementbenchmark, as well as on electronic health records. Our results suggest thatnonidentifiable hidden confounding bias can be mitigated to improve offline RLsolutions in practice.

Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding