English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Neural audio synthesis is an actively researched topic, having yielded a widerange of techniques that leverages machine learning architectures. GoogleMagenta elaborated a novel approach called Differential Digital SignalProcessing (DDSP) that incorporates deep neural networks with preconditioneddigital signal processing techniques, reaching state-of-the-art resultsespecially in timbre transfer applications. However, most of these techniques,including the DDSP, are generally not applicable in real-time constraints,making them ineligible in a musical workflow. In this paper, we present areal-time implementation of the DDSP library embedded in a virtual synthesizeras a plug-in that can be used in a Digital Audio Workstation. We focused ontimbre transfer from learned representations of real instruments to arbitrarysound inputs as well as controlling these models by MIDI. Furthermore, wedeveloped a GUI for intuitive high-level controls which can be used forpost-processing and manipulating the parameters estimated by the neuralnetwork. We have conducted a user experience test with seven participantsonline. The results indicated that our users found the interface appealing,easy to understand, and worth exploring further. At the same time, we haveidentified issues in the timbre transfer quality, in some components we did notimplement, and in installation and distribution of our plugin. The nextiteration of our design will address these issues. Our real-time MATLAB andJUCE implementations are available at https://github.com/SMC704/juce-ddsp andhttps://github.com/SMC704/matlab-ddsp , respectively.

Real-time Timbre Transfer and Sound Synthesis using DDSP