QC-StyleGAN: Quality Controllable Image Generation and Manipulation
Nov 28, 2022·
,,,,·
1 min read
Dat Viet Thanh Nguyen
Phong Tran The
Tan M. Dinh
Cuong Pham
Anh Tuan Tran
QC-StyleGAN generates and restores images across a continuous quality axis.Abstract
The introduction of high-quality image generation models, particularly the StyleGAN family, provides a powerful tool to synthesize and manipulate images. However, existing models are built upon high-quality (HQ) data as desired outputs, making them unfit for in-the-wild low-quality (LQ) images, which are common inputs for manipulation. In this work, we bridge this gap by proposing a novel GAN structure that allows for generating images with controllable quality. The network can synthesize various image degradation and restore the sharp image via a quality control code. Our proposed QC-StyleGAN can directly edit LQ images without altering their quality by applying GAN inversion and manipulation techniques. It also provides for free an image restoration solution that can handle various degradations, including noise, blur, compression artifacts, and their mixtures. Finally, we demonstrate numerous other applications such as image degradation synthesis, transfer, and interpolation.
Type
Publication
Advances in Neural Information Processing Systems (NeurIPS) 35
Why this problem
StyleGAN and its successors are trained on curated, high-quality datasets. That is fine for synthesis, but it breaks the moment you want to edit a real photo: GAN inversion pushes a blurry, noisy, or JPEG-compressed input onto a high-quality manifold, so the model silently “restores” the image while you were only trying to change the hairstyle. You lose the ability to make an edit that preserves the original look.
What we did
We added an explicit quality control code to the generator, so image content and image degradation become separately controllable. The same network can:
- synthesise images at any point along a continuous quality axis;
- invert and edit a low-quality image without changing its quality;
- restore a sharp image by simply moving the quality code — giving blind restoration across noise, blur, compression artifacts, and their mixtures, without training a restoration model;
- synthesise, transfer, and interpolate degradations between images.
Published at NeurIPS 2022; the work also received a NeurIPS Scholar Award. Code is available in the official repository.
