English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Physics-informed neural networks (PINNs) have lately received great attentionthanks to their flexibility in tackling a wide range of forward and inverseproblems involving partial differential equations. However, despite theirnoticeable empirical success, little is known about how such constrained neuralnetworks behave during their training via gradient descent. More importantly,even less is known about why such models sometimes fail to train at all. Inthis work, we aim to investigate these questions through the lens of the NeuralTangent Kernel (NTK); a kernel that captures the behavior of fully-connectedneural networks in the infinite width limit during training via gradientdescent. Specifically, we derive the NTK of PINNs and prove that, underappropriate conditions, it converges to a deterministic kernel that staysconstant during training in the infinite-width limit. This allows us to analyzethe training dynamics of PINNs through the lens of their limiting NTK and finda remarkable discrepancy in the convergence rate of the different losscomponents contributing to the total training error. To address thisfundamental pathology, we propose a novel gradient descent algorithm thatutilizes the eigenvalues of the NTK to adaptively calibrate the convergencerate of the total training error. Finally, we perform a series of numericalexperiments to verify the correctness of our theory and the practicaleffectiveness of the proposed algorithms. The data and code accompanying thismanuscript are publicly available at\url{https://github.com/PredictiveIntelligenceLab/PINNsNTK}.

When and why PINNs fail to train: A neural tangent kernel perspective