MSc Thesis defence by Yiran Zhang


Learned Regularizers in Variational Image Inpainting


Many problems in image processing and computer vision can be posed in the framework of inverse problem where solutions are obtained by minimizing an objective function, usually from a data fidelity term and a regularizer. Inpainting is now a classical one to be solved. The problem usually needs a suitable regularization, capable of enforcing some structure or selecting among the many completion choices. This thesis proposes approaches for this problem of restoring image: use neural networks either to formulate proper regularization or to replace regularization elements of optimization in classical minimizing schemes. On the one hand we think image inpainting is an important method to evaluate how well the network can understand and make use of the structural information in the image, then predict the missing structural information. On the other hand if we could use suitable optimization algorithms combined with proper regularizer to achieve better results, it would be a crucial technology to reduce heavy computation on training the networks.

This thesis consists of several parts, after an introduction, we discuss image inpainting as an inverse problem.

We illustrate it with TV inpaintings, and optimization schemes to solve it. Then we use DnCNN and DAEs as replacement for the TV proximal step, which is a denoising step. They inherently carry information about image structure and are expected to be more robust.

We also use Deep Learning techniques to replace TV by a better learned regularizer from VAEs and show its capabilities. We believe that the latter could be a systematic approach to build regularizer in image inverse problems.

Supervisor: François Lauze, DIKU

External examiner: Morten Pol Engell-Nørregård, Alexandra Institute