QC-StyleGAN: Quality Controllable Image Generation and Manipulation

Mon, 28 Nov 2022 00:00:00 +0000

Why this problem

StyleGAN and its successors are trained on curated, high-quality datasets. That is fine for synthesis, but it breaks the moment you want to edit a real photo: GAN inversion pushes a blurry, noisy, or JPEG-compressed input onto a high-quality manifold, so the model silently “restores” the image while you were only trying to change the hairstyle. You lose the ability to make an edit that preserves the original look.

What we did

We added an explicit quality control code to the generator, so image content and image degradation become separately controllable. The same network can:

synthesise images at any point along a continuous quality axis;
invert and edit a low-quality image without changing its quality;
restore a sharp image by simply moving the quality code — giving blind restoration across noise, blur, compression artifacts, and their mixtures, without training a restoration model;
synthesise, transfer, and interpolate degradations between images.

Published at NeurIPS 2022; the work also received a NeurIPS Scholar Award. Code is available in the official repository.

Image Restoration | Dat Viet Thanh Nguyen

QC-StyleGAN: Quality Controllable Image Generation and Manipulation

Why this problem

What we did