Neural Networks under Hamiltonian Constraints: A Comprehensive Review on Structural Evolution and Applications

Traditional algorithms and neural networks often suffer from energy drift in long-term predictions when modeling simple and fundamental natural laws, primarily due to the lack of inductive biases rooted in physical structures. By incorporating Hamiltonian mechanics into neural networks, these models leverage the conservation of energy and other desirable properties of Hamiltonian systems, leading to improved performance in tasks such as trajectory prediction, fundamental law learning, and image recognition. This study reviews the development and applications of neural networks constructed with Hamiltonian mechanics. We analyze two main categories: Hamiltonian Neural Networks (HNNs) and generative Hamiltonian neural networks, focusing on their architectural designs, data representations, energy conservation capabilities, loss function formulations, and choices of numerical integrators. Furthermore, we assess the advantages and challenges of these models in real-world applications. Hamiltonian-based neural networks demonstrate significant potential in preserving energy conservation and enhancing physical consistency. Future research should aim to optimize network architectures and parameter update mechanisms to improve both performance and interpretability. Additionally, exploring the integration of Hamiltonian neural networks with large language models may offer novel insights for tackling complex tasks and achieving more efficient learning.