MGFR: Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model
Keda Tao, Jinjin Gu*, Yulun Zhang, Xiucheng Wang and Nan Cheng, "Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model"
- 2025-1-22: Our paper has been accepted by ICLR 2025 (Spotlight)!.
- 2025-4-22: The new dataset we proposed-Reface-HQ-has been released.
Abstract: We introduce a novel Multi-modal Guided Real-World Face Restoration (MGFR) technique designed to improve the quality of facial image restoration from low-quality inputs. Leveraging a blend of attribute text prompts, high-quality reference images, and identity information, MGFR can mitigate the generation of false facial attributes and identities often associated with generative face restoration methods. By incorporating a dual-control adapter and a two-stage training strategy, our method effectively utilizes multi-modal prior information for targeted restoration tasks. We also present the Reface-HQ dataset, comprising over 21,000 high-resolution facial images across 4800 identities, to address the need for reference face training images. Our approach achieves superior visual quality in restoring facial details under severe degradation and allows for controlled restoration processes, enhancing the accuracy of identity preservation and attribute correction. Including negative quality samples and attribute prompts in the training further refines the model's ability to generate detailed and perceptually accurate images.
- Release Reface-HQ dataset.
- Release code and pretrained models.
We introduce a novel, high-quality dataset tailored for reference-based face image restoration tasks, which is now publicly accessible. The dataset comprises face images from over 4,800 unique identities, offering an expanded pool of training samples at higher image resolutions.
This project is based on ControlNet. Thanks for their awesome work.
If you have any questions, please feel free to contact with me at [email protected]