-
Notifications
You must be signed in to change notification settings - Fork 1
Description
The experiments cover the Latent Diffusion Models. However, I would like to raise a question for a discussion.
The training of the LDMs includes 2 steps: Step1. Training a VQ-VAE; Step2. Training a diffusion model in the latent space encoded by the encoder of the VQ-VAE.
Regarding to the released checkpoints by the LDM paper, the training of the VQ-VAE is on the OpenImage data set, and then the diffusion models are trained on the specific dataset, e.g., ImageNet. Therefore, the LDMs on ImageNet actually have an implicit prior on more knowledge than the ImageNet data set, since the LDMs are based the OpenImage-trained latent space.
Accordingly, in such an OoD detection task, it might not be appropriate to use such LDMs, since the in-distribution for training the LDMs actually covers more than the ImageNet distribution.