Schema Versions as Entrypoint Inputs Over vars#310
Merged
Conversation
CodeGat
added
type:enhancement
…hema path for potentially different schemas
…a paths to entrypoint workflows
CodeGat
force-pushed
the
239-schema-version-and-path-input
branch
from
aa15a37 to
a54f5d5
Compare
Merged
CodeGat
added a commit
that referenced
this pull request
Aug 11, 2025
* Injection of Modules over Checks (#295) * Add projection injection module * Add requirements for injection modules * Add tests for injection projection module, and inputs/requirements * Add prerelease injection module * Add tests for prerelease injection module * Run injection.projection and injection.prerelease modules * deploy-1-setup: Remove projection checks * Remove unneeded mergedeep requirement * Add module inits to folders * Add step for installing dependencies, update names for checkout steps * Used cd module prefix rather than custom working-directory so we can find the manifest * Added required input for prerelease modification, always print out updated manifest at end * Add back EOT * Update non-root-spec projections to namespaced `{name}/prX-Y/VERSION` format * Updated tests for namespaced non-root-spec projections * Update prerelease module argument from --root-spec-version to --version * Pass in sys.argv to parse_args, add tests * Refactor workflow to split generation of root spec and packages * Rename projections script to modules * Add inject_includes function * For test inputs, comment what is expected * Reordered functions * Updated tests after code updates * Apply suggestions from code review Co-authored-by: Aidan Heerdegen <[email protected]> Signed-off-by: Tommy Gatti <[email protected]> * Add root_spec to includes set, but remove it later when alphabetically sorting the includelist since it always comes first * Updated output of manifest * Ran black * Added getter module for accessing various parts of a spack manifest * Update `[test_]modules.py` to use new getter module * Update `[test_]prerelease.py` to use getter module, update deploy-2-start to remove --root-spec arg * Ran black again * Raise NotImplementedErrors, add comments for successful matches * Remove comments of old code * Apply suggestions from code review Co-authored-by: Aidan Heerdegen <[email protected]> Signed-off-by: Tommy Gatti <[email protected]> * Add example to multi-target spec to test function * Parameterized tests * Fixes following more test cases * Converted manifest with no packages to fixture * Improved from_file classmethod tests * Parameterized valid and mistakenly invalid tests * Remove `root-spec` arg and instead get from manifest directly, update tests * Add new prerelease script flag `--keep-root-spec-intact`, that doesn't remove version information --------- Signed-off-by: Tommy Gatti <[email protected]> Co-authored-by: Aidan Heerdegen <[email protected]> * `vars` to File-Based Special Packages (#305) * Validate MDR config/packages.json * Pass info from MDR config/packages.json to module injection script * Update variable name to `vars.CONFIG_PACKAGES_SCHEMA_VERSION` * Change cwd relative to build-cd rather than model for scripts * Add output of packages to be injected/added to provenance DB * Update prerelease projection to `ROOT_SPEC/prX-Y/{name}/VERSION` * Update tests for new prerelease injection logic * Replace all internal refs with v6 * Dump quoted strings for prerelease projections, updated expected output * Schema Versions as Entrypoint Inputs Over `vars` (#310) * Change schema vars to required inputs, add optional spack manifest schema path for potentially different schemas * Add required schema version inputs, and optional spack manifest schema paths to entrypoint workflows * Get packages for provenance from an earlier step * Update prerelease injection to the form `ROOT_SPEC/.dependencies/PRX-Y/VERSION-{hash:7}` * Fixed up an unused variable * Remove `.` from `dependencies directory --------- Signed-off-by: Tommy Gatti <[email protected]> Co-authored-by: Aidan Heerdegen <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #239
Background
varsare difficult to find, can't be versioned, and apply to the whole repository. In the linked issue, this was a reason to get rid of them.Now, the versions for the various schema are instead passed in to the entrypoint workflow, giving provenance as to when they changed and PR/commit-level granularity.
The new inputs are:
spack-manifest-schema-path(Optional, defaults toau.org.access-nri/model/spack/environment/deployment): The path to the schema relative toACCESS-NRI/schema. Useful for using different schema depending on the type of software deployed - for example, models vs general software deployment viaspackspack-manifest-schema-version(Required): The version for the schema (underspack-manifest-schema-path) that validates spack manifests.config-versions-schema-version(Required): The version for the schema that validates theconfig/versions.jsonfile.config-packages-schema-version(Required): The version for the schema that validates theconfig/packages.jsonfile.Important
Because we are adding new, required inputs to the entrypoint workflows, this will be a MAJOR update.
The PR
Testing
Since we will be effectively retesting the same code, we will do the main test under the
dev-v6->v6PR.