Evolving multimodal controllers with HyperNEAT

Natural brains effectively integrate multiple sensory modalities and act upon the world through multiple effector types. As researchers strive to evolve more sophisticated neural controllers, confronting the challenge of multimodality is becoming increasingly important. As a solution, this paper presents a principled new approach to exploiting indirect encoding to incorporate multimodality based on the HyperNEAT generative neuroevolution algorithm called the multi-spatial substrate (MSS). The main idea is to place each input and output modality on its own independent plane. That way, the spatial separation of such groupings provides HyperNEAT an a priori hint on which neurons are associated with which that can be exploited from the start of evolution. To validate this approach, the MSS is compared with more conventional approaches to HyperNEAT substrate design in a multiagent domain featuring three input and two output modalities. The new approach both significantly outperforms conventional approaches and reduces the creative burden on the user to design the layout of the substrate, thereby opening formerly prohibitive multimodal problems to neuroevolution.