Space Depends On Time: Informational Asymmetries in Visual and Auditory Short-Term Memory

Sensory modalities vary in adeptness for spatial and temporal information domains (Welch & Warren, 1980). Recent work suggests that attention and short-term memory (STM) recruit this variability (Michalka et al., submitted). Here, we investigate the relationships among visual and auditory modalities, and spatial and temporal STM. We developed stimuli comprising short sequences of visual events (instantaneous image changes) or auditory events (50ms complex tones). Each event within a sequence had a unique spatial location and a unique inter-event interval. These stimuli were used in a STM change-detection task. On each trial, two successive sequences were presented; a change could occur among the locations, the intervals, both, or neither. Subjects attended to either space (the sequence of locations), or time (the sequence of inter-event intervals), and reported whether the patterns of locations or intervals were identical. Each subject completed blocks of unimodal (both sequences presented in the same modality) and crossmodal (sequence 1 visual and sequence 2 auditory, or vice versa) trials for both tasks. We found a strong modality appropriateness effect, with best temporal performance on unimodal auditory trials, and best spatial performance on unimodal visual trials. The order of modalities on crossmodal trials mattered for space (benefit for visual sequence 1) but not for time, supporting a domain recruitment account of spatial STM. We also investigated cross-domain interactions by measuring whether instability of spatial location affected change detection for intervals, or vice versa. Changes in timing from sequence 1 to sequence 2 substantially impaired change detection for locations, while changes in locations did not impair change detection for intervals. These results suggest that spatial and temporal STM are asymmetrically related, such that timing information facilitates monitoring a series of locations, but spatial knowledge is unnecessary when monitoring a series of intervals. Meeting abstract presented at VSS 2015.