Foreground Segmentation and High-Resolution Depth Map Generation Using a Time-of-Flight Depth Camera

In this paper, we propose a foreground extraction and depth map generation method using a time-of-flight (TOF) depth camera. Although, the TOF depth camera captures the scene's depth information in real-time, it has a built-in noise and distortion. Therefore, we perform several preprocessing steps such as image enhancement, segmentation, and 3D warping, and then use the TOF depth data to generate the depth-discontinuity regions. Then, we extract the foreground object and generate the depth map as of the color image. The experimental results show that the proposed method efficiently generates the depth map even for the object boundary and textureless regions. * : , yunsuk@gist.ac.kr, ° : , hoyo@gist.ac.kr, :KICS2012-07-303, :2012 7 4, :2012 9 5 I. 3 , . (multi-view image) , , , [1] . '12-09 Vol.37C No.09 752 . , . , . . (stereo matching) [2] . (disparity) , , , . , . Time-of-Flight (TOF) . , . , , . [3-6] . , , . , . , , . II. TOF TOF . Mesa Imaging SR4000 1 . , 0.3m, 5.0m, 54 . SR4000 1 , 8bit (depth image) (intensity image) , 176x144 . (illumination cover) (optical filter) (depth image) (intensity image) 1. Fig. 1. Depth camera and captured image , . [3, 4], [5] , [6], . , . III. . 2 . , . , , / 753 . 2. Fig. 2. Proposed depth map generation method 3.1. . . 1(b) , [7] . (camera calibration) [8] . (intrinsic) 3 (orientation) , (translation) , (projection matrix) . 3 (warping) . 3.2. . , . , , . . , , . . 3 Canny . . 3. Fig. 3. Depth camera image and edge information 3 . 3 . 4 , 3 , 3 . 4. 3 Fig. 4. 3D warping of depth image , 3 , , , . . , , 5 . 5(a) 5(b) , . 5(a) '12-09 Vol.37C No.09 754 . Grab-cut [9] . , , 5(a) 6(a) , Grab-cut . 6(b) . . (a) (b) 5. Fig. 5. Depth discontinuity region generation (a) (b) 6. Grab-cut Fig. 6. Foreground object segmentation using Grab-cut 3.3 6 . 7 3 . . . 7. 3 Fig. 7. 3D warping of depth camera image 6(b) , . 7 3 . d(x+i, y+j) d(x, y) (1) , , . (1) n 0 . d(x+i, y+j) . c(x, y) . k . if (2) , .