HyperGANStrument: Instrument Sound Synthesis and Editing with Pitch-Invariant Hypernetworks

GANStrument, exploiting GANs with a pitch-invariant feature extractor andinstance conditioning technique, has shown remarkable capabilities insynthesizing realistic instrument sounds. To further improve the reconstructionability and pitch accuracy to enhance the editability of user-provided sound,we propose HyperGANStrument, which introduces a pitch-invariant hypernetwork tomodulate the weights of a pre-trained GANStrument generator, given a one-shotsound as input. The hypernetwork modulation provides feedback for the generatorin the reconstruction of the input sound. In addition, we take advantage of anadversarial fine-tuning scheme for the hypernetwork to improve thereconstruction fidelity and generation diversity of the generator. Experimentalresults show that the proposed model not only enhances the generationcapability of GANStrument but also significantly improves the editability ofsynthesized sounds. Audio examples are available at the online demo page.