Skip to content

Porting of MedNIST DDPM model-zoo from MONAI Generative#721

Merged
ericspod merged 17 commits intoProject-MONAI:devfrom
virginiafdez:mednist_ddpm
Feb 5, 2025
Merged

Porting of MedNIST DDPM model-zoo from MONAI Generative#721
ericspod merged 17 commits intoProject-MONAI:devfrom
virginiafdez:mednist_ddpm

Conversation

@virginiafdez
Copy link
Copy Markdown
Contributor

Description

Initial porting from the Generative model-zoo of the mednist-ddpm model (https://github.com/Project-MONAI/GenerativeModels/tree/main/model-zoo/models/mednist_ddpm/bundle).

The bundle files are tested within the docs/2d_ddpm_bundle_tutorial.ipynb

Status

Ready

Please ensure all the checkboxes:

  • Update version and changelog in metadata.json if changing an existing bundle.
  • Please ensure the naming rules in config files meet our requirements (please refer to: CONTRIBUTING.md).
  • Ensure versions of packages such as monai, pytorch and numpy are correct in metadata.json.
  • Avoid using path that contains personal information within config files (such as use /home/your_name/ for "bundle_root").

virginia and others added 5 commits January 12, 2025 14:39
…erify_bundle to pass the check for model.pt, since the requirement for two models (autoencoder and diffusion_model) makes sense for them to keep their specific names.

Modification of inference.json to add dummy attributes to pass the ConfigWorkflow check.
Modification of large_files.yml so that models are .pt and not .pth.
@virginiafdez virginiafdez changed the title Mednist ddpm Porting of MedNIST DDPM model-zoo from MONAI Generative Jan 13, 2025
@KumoLiu
Copy link
Copy Markdown
Collaborator

KumoLiu commented Jan 24, 2025

/build

@KumoLiu
Copy link
Copy Markdown
Collaborator

KumoLiu commented Jan 24, 2025

[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,107 - INFO - Downloaded: models/mednist_ddpm/models/model.pt
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,151 - INFO - Verified 'model.pt', md5: 02fd8c8e8ed5f7cda5deeed72b69f4f1.
[2025-01-24T16:06:13.254Z] directory is verified correctly.
[2025-01-24T16:06:13.254Z] version and changelog are verified correctly.
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,153 - INFO - --- input summary of monai.bundle.scripts.verify_metadata ---
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,153 - INFO - > meta_file: 'models/mednist_ddpm/configs/metadata.json'
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,153 - INFO - > filepath: 'models/mednist_ddpm/eval/schema.json'
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,153 - INFO - ---
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,154 - INFO - Expected md5 is None, skip md5 check for file models/mednist_ddpm/eval/schema.json.
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,154 - INFO - File exists: models/mednist_ddpm/eval/schema.json, skipped downloading.
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - metadata is verified with no error.
[2025-01-24T16:06:13.254Z] metadata format is verified correctly.
[2025-01-24T16:06:13.254Z] properties are verified correctly.
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - --- input summary of monai.bundle.scripts.verify_net_in_out ---
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - > net_id: 'network_def'
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - > meta_file: 'models/mednist_ddpm/configs/metadata.json'
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - > config_file: 'models/mednist_ddpm/configs/inference.yaml'
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - > bundle_root: 'models/mednist_ddpm'
[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - ---
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] Detected deprecated name 'optional_packages_version' in configuration file, replacing with 'required_packages_version'.
[2025-01-24T16:06:13.254Z] Traceback (most recent call last):
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/reference_resolver.py", line 129, in _resolve_one_item
[2025-01-24T16:06:13.254Z]     item = look_up_option(id, self.items, print_all_options=False, default=kwargs.get("default", "no_default"))
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/utils/module.py", line 141, in look_up_option
[2025-01-24T16:06:13.254Z]     raise ValueError(f"Unsupported option '{opt_str}', " + supported_msg)
[2025-01-24T16:06:13.254Z] ValueError: Unsupported option 'network_def', 
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] The above exception was the direct cause of the following exception:
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] Traceback (most recent call last):
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/scripts.py", line 1233, in verify_net_in_out
[2025-01-24T16:06:13.254Z]     net = parser.get_parsed_content(key).to(device_)
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/config_parser.py", line 290, in get_parsed_content
[2025-01-24T16:06:13.254Z]     return self.ref_resolver.get_resolved_content(id=id, **kwargs)
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/reference_resolver.py", line 193, in get_resolved_content
[2025-01-24T16:06:13.254Z]     return self._resolve_one_item(id=id, **kwargs)
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/reference_resolver.py", line 131, in _resolve_one_item
[2025-01-24T16:06:13.254Z]     raise KeyError(f"id='{id}' is not found in the config resolver.") from err
[2025-01-24T16:06:13.254Z] KeyError: "id='network_def' is not found in the config resolver."
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] The above exception was the direct cause of the following exception:
[2025-01-24T16:06:13.254Z] 
[2025-01-24T16:06:13.254Z] Traceback (most recent call last):
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 359, in <module>
[2025-01-24T16:06:13.254Z]     verify(bundle, models_path, mode)
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 340, in verify
[2025-01-24T16:06:13.254Z]     verify_data_shape(bundle_path, net_id, config_file)
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 194, in verify_data_shape
[2025-01-24T16:06:13.254Z]     verify_net_in_out(
[2025-01-24T16:06:13.254Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/scripts.py", line 1235, in verify_net_in_out
[2025-01-24T16:06:13.254Z]     raise KeyError(f"Failed to verify due to missing expected key in the config: {key}.") from e
[2025-01-24T16:06:13.254Z] KeyError: 'Failed to verify due to missing expected key in the config: network_def.'

cc @virginiafdez @ericspod

@KumoLiu
Copy link
Copy Markdown
Collaborator

KumoLiu commented Jan 24, 2025

@ericspod
Copy link
Copy Markdown
Member

[2025-01-24T16:06:13.254Z] 2025-01-24 16:06:13,162 - INFO - > config_file: 'models/mednist_ddpm/configs/inference.yaml'

The issue here is that the CICD test isn't loading both common.yaml and inference.yaml I think? It needs common.yaml for the definitions, maybe we should just copy those definitions into the other files for now.

@KumoLiu
Copy link
Copy Markdown
Collaborator

KumoLiu commented Jan 27, 2025

verify_net_in_out

Yes, for now CICD didn't take the common.yaml into consideration. We may need enhance the CICD or directly copy the things out for now.
https://github.com/Project-MONAI/model-zoo/blob/dev/ci/verify_bundle.py#L334

@ericspod
Copy link
Copy Markdown
Member

verify_net_in_out

Yes, for now CICD didn't take the common.yaml into consideration. We may need enhance the CICD or directly copy the things out for now. https://github.com/Project-MONAI/model-zoo/blob/dev/ci/verify_bundle.py#L334

I spoke to @virginiafdez and we'll copy the contents of common.yaml into the appropriate files, and we'll revisit specifying these things in the bundle format and updating the checks.

@KumoLiu
Copy link
Copy Markdown
Collaborator

KumoLiu commented Feb 3, 2025

/build

@KumoLiu
Copy link
Copy Markdown
Collaborator

KumoLiu commented Feb 5, 2025

/build

@ericspod ericspod merged commit 713255b into Project-MONAI:dev Feb 5, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants