Product of Gaussian Mixture Diffusion Models

In this work we tackle the problem of estimating the density $ f_X $ of arandom variable $ X $ by successive smoothing, such that the smoothed randomvariable $ Y $ fulfills the diffusion partial differential equation $(\partial_t - \Delta_1)f_Y(\,\cdot\,, t) = 0 $ with initial condition $f_Y(\,\cdot\,, 0) = f_X $. We propose a product-of-experts-type model utilizingGaussian mixture experts and study configurations that admit an analyticexpression for $ f_Y (\,\cdot\,, t) $. In particular, with a focus on imageprocessing, we derive conditions for models acting on filter-, wavelet-, andshearlet responses. Our construction naturally allows the model to be trainedsimultaneously over the entire diffusion horizon using empirical Bayes. We shownumerical results for image denoising where our models are competitive whilebeing tractable, interpretable, and having only a small number of learnableparameters. As a byproduct, our models can be used for reliable noise levelestimation, allowing blind denoising of images corrupted by heteroscedasticnoise.