English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Trajectory Optimization (TO) and Reinforcement Learning (RL) are powerful andcomplementary tools to solve optimal control problems. On the one hand, TO canefficiently compute locally-optimal solutions, but it tends to get stuck inlocal minima if the problem is not convex. On the other hand, RL is typicallyless sensitive to non-convexity, but it requires a much higher computationaleffort. Recently, we have proposed CACTO (Continuous Actor-Critic withTrajectory Optimization), an algorithm that uses TO to guide the exploration ofan actor-critic RL algorithm. In turns, the policy encoded by the actor is usedto warm-start TO, closing the loop between TO and RL. In this work, we presentan extension of CACTO exploiting the idea of Sobolev learning. To make thetraining of the critic network faster and more data efficient, we enrich itwith the gradient of the Value function, computed via a backward pass of thedifferential dynamic programming algorithm. Our results show that the newalgorithm is more efficient than the original CACTO, reducing the number of TOepisodes by a factor ranging from 3 to 10, and consequently the computationtime. Moreover, we show that CACTO-SL helps TO to find better minima and toproduce more consistent results.

CACTO-SL: Using Sobolev Learning to improve Continuous Actor-Critic with Trajectory Optimization