English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We revisit the inverted pendulum problem with the goal of understanding andcomputing the true optimal value function. We start with an observation thatthe true optimal value function must be nonsmooth ($i.e.$, not globally $C^1$)due to the symmetry of the problem. We then give a result that can certify theoptimality of a candidate $\textit{piece-wise}$ $C^1$ value function. Further,for a candidate value function obtained via numerical approximation, we providea bound of suboptimality based on its Hamilton-Jacobi-Bellman (HJB) equationresiduals. Inspired by Holzhuter (2004), we then design an algorithm thatsolves backward the Pontryagin's minimum principle (PMP) ODE from terminalconditions provided by the locally optimal LQR value function. This numericalprocedure leads to a piece-wise $C^1$ value function whose nonsmooth regioncontains periodic $\textit{spiral lines}$ and smooth regions attain HJBresiduals about $10^{-4}$, hence certified to be the optimal value function upto minor numerical inaccuracies. This optimal value function checks the powerof optimality: (i) it sits above a polynomial lower bound; (ii) its inducedcontroller globally swings up and stabilizes the pendulum, and (iii) attainslower trajectory cost than baseline methods such as energy shaping, modelpredictive control (MPC), and proximal policy optimization (with MPC attainingalmost the same cost). We conclude by distilling the optimal value functioninto a simple neural network.

On the Nonsmooth Geometry and Neural Approximation of the Optimal Value Function of Infinite-Horizon Pendulum Swing-up