Open AccessNoiseBandNet: Controllable Time-Varying Neural Synthesis of Sound Effects Using Filterbanks
Author(s) -
Adrian Barahona-Rios,
Tom Collins
Publication year - 2024
Publication title -
ieee/acm transactions on audio, speech, and language processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.916
H-Index - 56
eISSN - 2329-9304
pISSN - 2329-9290
DOI - 10.1109/taslp.2024.3364616
Subject(s) - signal processing and analysis , computing and processing , communication, networking and broadcast technologies , general topics for engineers
Controllable neural audio synthesis of sound effects is a challenging task due to the potential scarcity and spectro-temporal variance of the data. Differentiable digital signal processing (DDSP) synthesisers have been successfully employed to model and control musical and harmonic signals using relatively limited data and computational resources. Here we propose NoiseBandNet, an architecture capable of synthesising and controlling sound effects by filtering white noise through a filterbank, thus going further than previous systems that make assumptions about the harmonic nature of sounds. We evaluate our approach via a series of experiments, modelling footsteps, thunderstorm, pottery, knocking, and metal sound effects. Comparing NoiseBandNet audio reconstruction capabilities to four variants of the DDSP-filtered noise synthesiser, NoiseBandNet scores higher in nine out of ten evaluation categories, establishing a flexible DDSP method for generating time-varying, inharmonic sound effects of arbitrary length with both good time and frequency resolution. Finally, we introduce some potential creative uses of NoiseBandNet, by generating variations, performing loudness transfer, and by training it on user-defined control curves.