English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Q-learning has long been one of the most popular reinforcement learningalgorithms, and theoretical analysis of Q-learning has been an active researchtopic for decades. Although researches on asymptotic convergence analysis ofQ-learning have a long tradition, non-asymptotic convergence has only recentlycome under active study. The main goal of this paper is to investigate newfinite-time analysis of asynchronous Q-learning under Markovian observationmodels via a control system viewpoint. In particular, we introduce adiscrete-time time-varying switching system model of Q-learning withdiminishing step-sizes for our analysis, which significantly improves recentdevelopment of the switching system analysis with constant step-sizes, andleads to \(\mathcal{O}\left( \sqrt{\frac{\log k}{k}} \right)\) convergence ratethat is comparable to or better than most of the state of the art results inthe literature. In the mean while, a technique using the similarlytransformation is newly applied to avoid the difficulty in the analysis posedby diminishing step-sizes. The proposed analysis brings in additional insights,covers different scenarios, and provides new simplified templates for analysisto deepen our understanding on Q-learning via its unique connection todiscrete-time switching systems.

Finite-Time Analysis of Asynchronous Q-learning under Diminishing Step-Size from Control-Theoretic View