Audio-Visual Cross-Modal Association in Color and Sound Perception

Previous studies have shown that a similar straightforward relationship exists between lightness and pitch in synaesthetes and nonsynaesthetes (Mark, 1974; Hubbard, 1996; Ward et al., 2006). These results indicated that nonsynaesthetes have similar lightness-pitch mapping with synaesthetes. However, in their experimental paradigm such a similarity seems to stem from not so much sensory phenomena as high-level memory associations in nonsynaesthetes. So, we examined what process in perception and/or cognition relates lightness-pitch synaesthesia-like phenomena in nonsynaethetes. In our study we targeted perceived colors (luminance) per se rather than the imagery of color (luminance) via memory association. Results indicated that the performance in color selection were affected by task-irrelevant stimuli (auditory information), but there was little significant correlation between color and auditory stimuli (pitch) in simple color conditions. However, in subjective figures conditions, results showed a different tendency and partly showed correlations. Recent work indicates synaesthesia needs selective attention (Rich et al., 2010) and some research shows perception of subjective contours need attention (Gurnsey, 1996). We conjecture that lightness-pitch synaesthesia-like phenomena may need some kind of attention or the other higher brain activity (eg, memory association, cognition)