Premium
STALP: Style Transfer with Auxiliary Limited Pairing
Author(s) -
Futschik D.,
Kučera M.,
Lukáč M.,
Wang Z.,
Shechtman E.,
Sýkora D.
Publication year - 2021
Publication title -
computer graphics forum
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.578
H-Index - 120
eISSN - 1467-8659
pISSN - 0167-7055
DOI - 10.1111/cgf.142655
Subject(s) - stylized fact , computer science , translation (biology) , consistency (knowledge bases) , artificial intelligence , image (mathematics) , set (abstract data type) , transfer (computing) , style (visual arts) , pairing , key (lock) , computer vision , pattern recognition (psychology) , physics , superconductivity , quantum mechanics , parallel computing , history , biochemistry , chemistry , computer security , macroeconomics , archaeology , messenger rna , economics , gene , programming language
Abstract We present an approach to example‐based stylization of images that uses a single pair of a source image and its stylized counterpart. We demonstrate how to train an image translation network that can perform real‐time semantically meaningful style transfer to a set of target images with similar content as the source image. A key added value of our approach is that it considers also consistency of target images during training. Although those have no stylized counterparts, we constrain the translation to keep the statistics of neural responses compatible with those extracted from the stylized source. In contrast to concurrent techniques that use a similar input, our approach better preserves important visual characteristics of the source style and can deliver temporally stable results without the need to explicitly handle temporal consistency. We demonstrate its practical utility on various applications including video stylization, style transfer to panoramas, faces, and 3D models.