Shulou(Shulou.com)11/24 Report--

CTOnews.com, Nov. 4, Tencent AI Lab recently launched a progressive conditional diffusion model (PCDMs), which has made a major breakthrough in pose-guided character image synthesis.

PCDMs consists of three key stages: a priori conditional diffusion model, a repair conditional diffusion model and a perfect conditional diffusion model, which solves the problem of pose inconsistency between the source image and the target image, as well as the challenge of generating high quality and realistic images.

The indicators of PCDMs on DeepFashion and Market1501 datasets are significantly better than other SOTA methods, and the SSIM index on the small-scale dataset Market1501 (128 / 64) is the highest 0.3169, 3.8% higher than the second PIDM.

In the first stage of the prior conditional diffusion model, when the source image and pose coordinates are given as conditions, the prior conditional diffusion model uses a transformation network to predict the global features of the target pose.

In the second stage of the repair conditional diffusion model, we further improve the global features of the first stage and establish a dense correspondence between the source image and the target image, which ensures alignment across multiple dimensions (including images, poses and features). It is very important to achieve realistic results.

In the third stage of improving the conditional diffusion model: after the initial coarse-grained target image is generated in the previous stage, the thinning conditional diffusion model is involved to improve the image quality and texture details.

This stage uses the previously generated coarse-grained image as a condition to further improve image fidelity and ensure texture consistency, which involves modifying the first convolution layer and using an image encoder to extract features from the source image. The cross-attention mechanism is used to inject texture features into the network to facilitate texture repair and detail enhancement.

CTOnews.com encloses here the address of the paper: https://arxiv.org/pdf/2310.06313.pdf

GitHub address: https://github.com/muzishen/PCDMs

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.