Revisiting Horn and Schunck: Interpretation as Gauss-Newton Optimisation

We revisit the classic Horn and Schunck optical flow method, with focus on its interpretation as a Gauss-Newton (GN) minimisation of a non-linear energy. This is in contrast to the traditional derivation by linearisation of the brightness constancy assumption. The proposed interpretation provides a much simpler derivation and a better theoretical understanding of the method, and allows for its variations, by casting it in the least-squares optimisation framework. An important resulting implication is that ‐ in contrast to popular belief ‐ the incremental version of Horn and Schunck actually minimises a non-linear energy. We emphasise this finding by demonstrating the equivalence of incremental Horn and Schunck to several methods committed to minimising the nonlinearised energy. Furthermore, we analyse the effect of GN for motion estimation in comparison to methods based on gradient descent. Several examples demonstrate the practical applications of the proposed interpretation. We specify the class of difference measures with sparse Jacobians of the error term as the one which can be efficiently treated in the Horn and Schunck framework. For extension to arbitrary difference measures, we propose a modification based on the analysis of GN for motion estimation, and the concept of preconditioning. We further discuss a modification resulting in decoupled linear systems and the use of compositional updates.