Posteriori Reconstruction and Denoising for Path Tracing

Path tracing is a well-established technique for photo-realistic rendering to simulate light path transport. This method has been widely adopted in visual effects industries to generate high quality synthetic images requiring a large number of samples and a long computation time. Due to the high cost to produce the final output, intermediate previsualization of path tracing is in high demand from production artists to detect errors in the early stage of rendering. But visualizing intermediate results of path tracing is also challenging since the synthesized image with limited samples or improper sampling usually suffers from distracting noise. The ideal solution would be to provide a highly plausible intermediate result in the early stages of rendering, using a small fraction of samples, and apply a posteriori manner to approximate the ground truth.  In this thesis, this issue is addressed by providing several efficient posteriori reconstructions and denoising technique for previsualization of pa-th tracing. Firstly, we address the problem for the recovery of the missing values to construct low rank matrices for incomplete images including missing pixel, missing sub-pixel, and multi-frame scenarios. A novel approach utilizing a convolutional neural network which provides fast precompletion for initializing missing values, and subsequent weighted nuclear norm minimization with a parameter adjustment strategy efficiently recovers missing values even in high frequency details. The result shows better visual quality compared to the recent methods including compressed sensing based reconstruction.  Furthermore, to mitigate the computation budgets of our new approac-h, we extend our method by applying a block Toeplitz structure forming a low-rank matrix for pixel recovery, and tensor structure for multi-frame recovery. In this manner, the reconstruction time can be significantly reduced. Besides that, by exploiting temporal coherence of multi-frame with a tensor structure, we demonstrate an improvement in the overall recovery quality compared to our previous approach.  Our recovery methods provide satisfying solution but still require plen-ty of rendering time at prior stage compared with denoising solutions. Finally, we introduce a novel filter for denoising based on convolutional neural network, to address the problem as conventional denoising approach for rendered images. Unlike a plain CNN that applies fixed kernel size in each layer, we propose a multi-scale residual network with various auxiliary scene features to leverage a new efficient denoising filter for path tracing. Our experimental results show on par or better denoising quality compare to state-of-the-art path tracing denoiser.