Skip to content

issues on the LDM experiments. #1

@fanghenshaometeor

Description

@fanghenshaometeor

The experiments cover the Latent Diffusion Models. However, I would like to raise a question for a discussion.

The training of the LDMs includes 2 steps: Step1. Training a VQ-VAE; Step2. Training a diffusion model in the latent space encoded by the encoder of the VQ-VAE.
Regarding to the released checkpoints by the LDM paper, the training of the VQ-VAE is on the OpenImage data set, and then the diffusion models are trained on the specific dataset, e.g., ImageNet. Therefore, the LDMs on ImageNet actually have an implicit prior on more knowledge than the ImageNet data set, since the LDMs are based the OpenImage-trained latent space.

Accordingly, in such an OoD detection task, it might not be appropriate to use such LDMs, since the in-distribution for training the LDMs actually covers more than the ImageNet distribution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions