Effects of Visual Transformation on Object Manipulation in 3D Space

People can effortlessly manipulate the cursor on 2D screen without seeing their hands. What happens in the 3D case? This study is to examine how visual feedback influences the performance of object manipulation in 3D space. In our experiments virtual 3D disks in different orientations were sequentially shown to subjects wearing stereo shutter glasses. Subjects' task was to touch the disks from designated directions with the stylus of a force-feedback device as fast as possible. Three conditions of visual feedbacks were set: (1) the subjects looked down at their hand and the virtual stylus was shown at the location of the actual stylus using a mirror; (2) the subjects looked straight forward and the virtual stylus was translated to the height of eye position; and (3) the subjects looked straight forward as in condition (2) whereas the virtual stylus was shown from the viewpoint of condition (1). The average performance time of condition (1) was shortest whereas that of (3) was in between. The results suggest that mental imagery of hand movement may be constructed from the viewpoint when looking at the hand in current posture, and used for manipulating objects in 3D space when the hand is not visible