Globally Optimal DLS Method for PnP Problem with Cayley parameterization

The perspective-n-point (PnP) problem, which estimates 3D rotation and translation of a calibrated camera from n pairs of known 3D points and corresponding 2D image points, is a classical problem but still fundamental in the computer vision community. It is well studied that the PnP problem can be solved by at least three points [1]. If n ≥ 4, the PnP problem becomes a nonlinear problem where the number of the solutions depend on n and the shape of the scene. This paper proposes an efficient, scalable, and globally optimal DLS method parameterized by Cayley representation, which has been regarded as a unsuitable parametrization due to its singularity. First we derive a new optimality condition without Lagrange multipliers. Letting pi = [xi,yi,zi] be an i-th 3D point and mi = [ui,vi,1] be the corresponding calibrated image point in homogeneous coordinates, the PnP problem can be formulated as a nonlinear optimization