English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

A novel reinforcement learning scheme to synthesize policies forcontinuous-space Markov decision processes (MDPs) is proposed. This schemeenables one to apply model-free, off-the-shelf reinforcement learningalgorithms for finite MDPs to compute optimal strategies for the correspondingcontinuous-space MDPs without explicitly constructing the finite-stateabstraction. The proposed approach is based on abstracting the system with afinite MDP (without constructing it explicitly) with unknown transitionprobabilities, synthesizing strategies over the abstract MDP, and then mappingthe results back over the concrete continuous-space MDP with approximateoptimality guarantees. The properties of interest for the system belong to afragment of linear temporal logic, known as syntactically co-safe lineartemporal logic (scLTL), and the synthesis requirement is to maximize theprobability of satisfaction within a given bounded time horizon. A keycontribution of the paper is to leverage the classical convergence results forreinforcement learning on finite MDPs and provide control strategies maximizingthe probability of satisfaction over unknown, continuous-space MDPs whileproviding probabilistic closeness guarantees. Automata-based reward functionsare often sparse; we present a novel potential-based reward shaping techniqueto produce dense rewards to speed up learning. The effectiveness of theproposed approach is demonstrated by applying it to three physical benchmarksconcerning the regulation of a room's temperature, control of a road trafficcell, and of a 7-dimensional nonlinear model of a BMW 320i car.

Formal Controller Synthesis for Continuous-Space MDPs via Model-Free Reinforcement Learning