CHESS—Calibrating the Hand-Eye Matrix With Screw Constraints and Synchronization

Hand-eye calibration is a classic problem in robotics that aims to find the transformation between two rigidly attached reference frames, usually a camera and a robot end-effector or a motion tracker. Most hand-eye calibration techniques require two data streams, one containing the eye (camera) motion and the other containing the hand (robot/tracker) motion, and the classic hand-eye formulation assumes that both data streams are fully synchronized. However, different motion capturing devices and cameras often have variable capture rates and timestamps that cannot always be easily triggered in sync. Although probabilistic approaches have been proposed to solve for nonsynchronized data streams, they are not able to deal with different capture rates. We propose a new approach for unsynchronized hand-eye calibration that is able to deal with different capture rates and time delays. Our method interpolates and resamples the signal with the lowest capture rate in a way that is consistent with the twist motion constraints of the hand-eye problem. Cross-correlation techniques are then used to produce two fully synchronized data streams that can be used to solve the hand-eye problem with classic methods. In our experimental section, we show promising validation results on simulation data and also on real data obtained from a robotic arm holding a camera.