English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, based on the spirit of Fitted Q-Iteration (FQI), we propose aGauss-Newton Temporal Difference (GNTD) method to solve the Q-value estimationproblem with function approximation. In each iteration, unlike the original FQIthat solves a nonlinear least square subproblem to fit the Q-iteration, theGNTD method can be viewed as an \emph{inexact} FQI that takes only oneGauss-Newton step to optimize this subproblem, which is much cheaper incomputation. Compared to the popular Temporal Difference (TD) learning, whichcan be viewed as taking a single gradient descent step to FQI's subproblem periteration, the Gauss-Newton step of GNTD better retains the structure of FQIand hence leads to better convergence. In our work, we derive the finite-samplenon-asymptotic convergence of GNTD under linear, neural network, and generalsmooth function approximations. In particular, recent works on neural TD onlyguarantee a suboptimal $\mathcal{\mathcal{O}}(\epsilon^{-4})$ samplecomplexity, while GNTD obtains an improved complexity of$\tilde{\mathcal{O}}(\epsilon^{-2})$. Finally, we validate our method viaextensive experiments in both online and offline RL problems. Our methodexhibits both higher rewards and faster convergence than TD-type methods,including DQN.

Provably Efficient Gauss-Newton Temporal Difference Learning Method with Function Approximation