Spatial Steerability of GANs via Self-Supervision from Discriminator

Generative models make huge progress to the photorealistic image synthesis inrecent years. To enable human to steer the image generation process andcustomize the output, many works explore the interpretable dimensions of thelatent space in GANs. Existing methods edit the attributes of the output imagesuch as orientation or color scheme by varying the latent code along certaindirections. However, these methods usually require additional human annotationsfor each pretrained model, and they mostly focus on editing global attributes.In this work, we propose a self-supervised approach to improve the spatialsteerability of GANs without searching for steerable directions in the latentspace or requiring extra annotations. Specifically, we design randomly sampledGaussian heatmaps to be encoded into the intermediate layers of generativemodels as spatial inductive bias. Along with training the GAN model fromscratch, these heatmaps are being aligned with the emerging attention of theGAN's discriminator in a self-supervised learning manner. During inference,users can interact with the spatial heatmaps in an intuitive manner, enablingthem to edit the output image by adjusting the scene layout, moving, orremoving objects. Moreover, we incorporate DragGAN into our framework, whichfacilitates fine-grained manipulation within a reasonable time and supports acoarse-to-fine editing process. Extensive experiments show that the proposedmethod not only enables spatial editing over human faces, animal faces, outdoorscenes, and complicated multi-object indoor scenes but also brings improvementin synthesis quality. Code, models, and demo video are available athttps://genforce.github.io/SpatialGAN/.