English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Physics-informed machine learning and inverse modeling require the solutionof ill-conditioned non-convex optimization problems. First-order methods, suchas SGD and ADAM, and quasi-Newton methods, such as BFGS and L-BFGS, have beenapplied with some success to optimization problems involving deep neuralnetworks in computational engineering inverse problems. However, empiricalevidence shows that convergence and accuracy for these methods remain achallenge. Our study unveiled at least two intrinsic defects of these methodswhen applied to coupled systems of partial differential equations (PDEs) anddeep neural networks (DNNs): (1) convergence is often slow with long plateausthat make it difficult to determine whether the method has converged or not;(2) quasi-Newton methods do not provide a sufficiently accurate approximationof the Hessian matrix; this typically leads to early termination (one of thestopping criteria of the optimizer is satisfied although the achieved error isfar from minimal). Based on these observations, we propose to use trust regionmethods for optimizing coupled systems of PDEs and DNNs. Specifically, wedeveloped an algorithm for second-order physics constrained learning, anefficient technique to calculate Hessian matrices based on computationalgraphs. We show that trust region methods overcome many of the defects andexhibit remarkable fast convergence and superior accuracy compared to ADAM,BFGS, and L-BFGS.

Trust Region Method for Coupled Systems of PDE Solvers and Deep Neural Networks