English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Recent advances in deep learning have enabled optimization of deep reactivepolicies (DRPs) for continuous MDP planning by encoding a parametric policy asa deep neural network and exploiting automatic differentiation in an end-to-endmodel-based gradient descent framework. This approach has proven effective foroptimizing DRPs in nonlinear continuous MDPs, but it requires a large number ofsampled trajectories to learn effectively and can suffer from high variance insolution quality. In this work, we revisit the overall model-based DRPobjective and instead take a minorization-maximization perspective toiteratively optimize the DRP w.r.t. a locally tight lower-bounded objective.This novel formulation of DRP learning as iterative lower bound optimization(ILBO) is particularly appealing because (i) each step is structurally easierto optimize than the overall objective, (ii) it guarantees a monotonicallyimproving objective under certain theoretical conditions, and (iii) it reusessamples between iterations thus lowering sample complexity. Empiricalevaluation confirms that ILBO is significantly more sample-efficient than thestate-of-the-art DRP planner and consistently produces better solution qualitywith lower variance. We additionally demonstrate that ILBO generalizes well tonew problem instances (i.e., different initial states) without requiringretraining.

Sample-efficient Iterative Lower Bound Optimization of Deep Reactive Policies for Planning in Continuous MDPs