Vision-Based Relative Pose Estimation of Non-cooperative Spacecraft using Time-of-Flight Camera

This paper deals with a vision-based pose estimation of a target using a time-of-flight (ToF) camera. In particular, a relative pose estimation algorithm for non-cooperative targets that lack markers or accessible sensor is proposed. The proposed method, called the CSAC-ICP algorithm, combines a feature-based coarse registration method, known as Corner Random Sample Consensus (CSAC), with the well-known Iterative Closest Point (ICP) algorithm to ensure both accuracy and robustness. Here, 3D corner points are adopted as primary features. The corner points are obtained by first converting 3D point clouds into 2D images from which line features are extracted and then re-projected into 3D lines, enabling 3D corner points computation. This reduces the computation on a large number of 3D points, ensuring real-time performance. Furthermore, an extended Kalman filter (EKF) is integrated with the proposed pose estimation algorithm in a loosely coupled manner, not only to continuously provide the target pose information for a moving target, but also to reduce measurement noises due to the motion. Because feature-based coarse registration takes advantage of the global optimization strategy, the proposed method can avoid the inherent pitfall of the standalone ICP method, in which the optimization process may fall into a local minimum and lead to incorrect pose calculation under certain conditions. Moreover, the proposed pose estimation scheme minimizes the dependency on the prior knowledge about the target object’s shape by taking advantage of the RANSAC-based registration. Numerical simulation is set up to validate the functionality of the algorithm using simulated 3D point clouds, confirming the accuracy performance under controlled target motion. In addition, the proposed algorithm is implemented and tested on an actual hardware platform, which demonstrates the feasibility of real-time relative pose estimation for non-cooperative targets within the space environment. Moreover, the proposed pose estimation scheme reduces reliance on prior knowledge of the target object’s geometry by using RANSAC-based registration. The simulation results show that the proposed CSAC-ICP algorithm achieves 30% faster computation and 50% higher accuracy compared to the standalone ICP method. Experimental validation further confirms that 3D corner features can be extracted at a rate of 30 frames per second. In addition, the pose estimation achieves an accuracy of up to 3 degrees for rotational motion and within 0.02 meters for translational motion along along all three axes, demonstrating the feasibility and superior performance of the proposed algorithm in real-world scenarios.