feat(collections): auto-generate collection.md with maturity filtering#1316
feat(collections): auto-generate collection.md with maturity filtering#1316
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1316 +/- ##
==========================================
- Coverage 87.94% 87.33% -0.61%
==========================================
Files 62 61 -1
Lines 9603 9428 -175
==========================================
- Hits 8445 8234 -211
- Misses 1158 1194 +36
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Advisory review — this PR is from a maintainer. Findings are informational only.
Review Summary
The implementation is well-structured and the PR meets quality standards. The description is clear, linked issue is present, type-of-change selections are accurate, and the 19 new Pester tests provide solid coverage of the new behavior. The utf8 → utf8NoBOM encoding fix in Set-ContentIfChanged is a correct bug fix. The maturity-filtering design is clean and consistent with existing channel logic.
Two advisory observations are raised below.
Issue Alignment ✅
Closes #1256. The PR delivers auto-generation of collection.md artifact tables with channel-based maturity filtering and sentinel-marker support, which matches the stated intent. The write-back behavior, backward-compatible marker validation, and plugin regeneration are all coherent.
PR Template Compliance ✅
| Section | Status |
|---|---|
| Description | ✅ Complete and detailed |
| Related Issue | ✅ Closes #1256 |
| Type of Change | ✅ New feature + Script/automation — both correct |
| Testing | ✅ Comprehensive (2002 tests, 0 failures, 19 new tests, lint & plugin generation) |
| Checklist | ✅ All required items checked |
Coding Standards
💡 Split-CollectionMdByMarkers — incomplete function decoration (advisory)
Every other function in Prepare-Extension.ps1 uses [CmdletBinding()], [OutputType()], and a full comment block (.DESCRIPTION, .PARAMETER, .OUTPUTS, .EXAMPLE). The PSScriptAnalyzer configuration explicitly sets PSProvideCommentHelp.ExportedOnly = $false, so the convention applies to private functions too. The inline comment above includes a suggested complete signature.
💡 Duplicate marker constants — DRY concern (advisory)
$script:CollectionMdBeginMarker and $script:CollectionMdEndMarker are now defined identically in both Prepare-Extension.ps1 and Validate-Collections.ps1. Since CollectionHelpers.psm1 is imported by both, exporting the constants from there would eliminate the duplication and make any future string change a single-point edit.
Code Quality ✅
No bugs, security issues, or logic errors found. The Split-CollectionMdByMarkers parsing logic handles all the edge cases covered by tests (no markers, partial markers, reversed markers, footer content). The write-back side-effect in New-CollectionReadme is clearly documented in the function's comment block. Test coverage is thorough.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none
katriendg
left a comment
There was a problem hiding this comment.
Left a few comments, I believe ensuring the plugin:generate takes care of generating the <collectionid>.collection.md file is a good approach, and we could have the Extension prepare regenerate what is needed based on maturity filtering.
Plugins always take everything without maturity filtering, so that in itself so the more I reflect on this, the more I think we need a generation for the plugins (the default), and regenerate one for stable/pre-release when packaging the extension.
Also this will impact the following documentation addition:
🟡 Comment 2 — Documentation gap: contributor guide has no mention of the two-zone collection.md convention
- File:
docs/contributing/ai-artifacts-common.mdand.github/copilot-instructions.md - Category: Documentation
- Severity: Should fix before merge
docs/contributing/ai-artifacts-common.md(the primary contributor guide for AI artifacts) extensivelydocuments*.collection.ymlstructure, maturity values, and plugin generation — but has no mention of:
- The
<!-- BEGIN AUTO-GENERATED ARTIFACTS -->/<!-- END AUTO-GENERATED ARTIFACTS -->marker format.- That
collection.mdhas two zones (hand-authored intro vs auto-generated table) and contributors must only edit the intro.- What command to run to refresh the artifact table after editing a YAML.
- How to seed a new collection with the correct marker structure.
.github/copilot-instructions.mdline 104 says "Only edit the intro zone; the artifact table is overwritten byplugin:generate" — this was added during the review session but the underlyingplugin:generatewiring gap (Comment 2 above) means this statement is currently inaccurate.A
### Collection Markdown Auto-Generationsubsection should be added todocs/contributing/ai-artifacts-common.mdexplaining the marker format, the two-zone rule, and the correct command to sync the artifact table. This section should be updated once Comment 2 is resolved so the documented command is accurate.
…kers - thread channel and maturity parameters through extension readme generation - add template markers to collection.md files with backward-compatible processing - add marker validation rules to collection validation pipeline - add unit tests for maturity filtering, marker handling, and marker validation 🔧 - Generated by Copilot
- expand New-CollectionReadme help with in-place update and marker behavior (IV-001) - add ValidateNotNullOrEmpty to AllowedMaturities parameter (IV-002) - centralize marker strings as script-scope constants in both scripts (IV-003) - remove unused Existing key from Split-CollectionMdByMarkers return (IV-004) - add duplicate-BEGIN-marker edge case test (IV-005) - add empty-string-throws test for Split-CollectionMdByMarkers (IV-006) - standardize Set-ContentIfChanged encoding to utf8NoBOM (IV-007) 🔧 - Generated by Copilot
…pers - move marker constants to CollectionHelpers.psm1 and export as module variables - move Split-CollectionMdByMarkers to CollectionHelpers.psm1 with full decorators - move Get-ArtifactDescription to CollectionHelpers.psm1 for reuse - add collection.md writeback block to Generate-Plugins.ps1 🔧 - Generated by Copilot
- regenerate all 14 collection.md files with structured artifact tables - replace hand-written summaries with auto-generated content between markers - tables include instructions, agents, prompts, skills per collection 📝 - Generated by Copilot
0fd25b0 to
1ec452c
Compare
There was a problem hiding this comment.
Advisory review — this PR is from a maintainer. Findings are informational only.
Review Summary
The refactor successfully closes #1256. The implementation is clean: maturity filtering is channel-driven, the utf8NoBOM encoding fix is correct, and the sentinel-marker write-back design is well-tested. The previous advisory's two blocking DRY concerns (missing [CmdletBinding()] on Split-CollectionMdByMarkers and duplicate marker constants) have both been addressed.
Three advisory observations remain — see inline comments.
Issue Alignment ✅
Closes #1256. Auto-generation of collection.md artifact tables with channel-based maturity filtering and sentinel marker support is fully implemented.
PR Template Compliance ✅
| Section | Status |
|---|---|
| Description | ✅ Detailed and accurate |
| Related Issue | ✅ Closes #1256 |
| Type of Change | ✅ New feature + Script/automation |
| Testing | ✅ 19 new Pester tests, 2002 total, 0 failures |
| Checklist | ✅ All required items checked |
Coding Standards
Three minor observations are raised as inline comments:
[OutputType([hashtable])]missing onSplit-CollectionMdByMarkers— the only function inCollectionHelpers.psm1without this annotation.- Duplicate allowed-maturity functions —
Get-AllowedMaturities(Prepare-Extension.ps1) andGet-AllowedCollectionMaturities(Generate-Plugins.ps1) are functionally identical; consolidation intoCollectionHelpers.psm1is a natural follow-on to the marker-constants refactor. - Artifact-table-building logic duplicated in
Generate-Plugins.ps1relative toNew-CollectionReadme— extracting a shared helper would reduce the maintenance surface.
Code Quality ✅
No bugs, logic errors, or security issues found. Maturity filtering is applied correctly in both code paths ($filteredCollection.items is pre-filtered by Select-CollectionItemsByChannel in the plugin path; AllowedMaturities is passed explicitly in the extension-readme path). The Split-CollectionMdByMarkers edge-case handling (reversed markers, partial markers, no markers, empty string) is well-covered by the new tests.
katriendg
left a comment
There was a problem hiding this comment.
Thanks, I think is should be good to merge. Just noticed you need to regenerate the plugins.
There was a problem hiding this comment.
Review Summary
This PR introduces auto-generation of collection.md artifact tables with channel-based maturity filtering and sentinel marker support. The overall direction is sound and the test coverage is thorough. However, there are required template compliance gaps and a functional code quality issue that need to be addressed before merging.
Issue Alignment
Linked issue #1256 is referenced as Closes #1256. The PR description and the scope of changes are self-consistent — the marker-based generation engine, -Channel parameter, maturity filtering, and new Pester tests all coherently address auto-generation of artifact tables. No obvious scope creep or missing requirements were identified based on the PR description.
PR Template Compliance
Two required sections from .github/PULL_REQUEST_TEMPLATE.md are absent from the PR body:
-
## Security Considerationssection is entirely missing. The template requires three checkboxes under this section:[ ] This PR does not contain any sensitive or NDA information[ ] Any new dependencies have been reviewed for security issues[ ] Security-related scripts follow the principle of least privilege
These checkboxes must be present and completed.
-
Two Required Automated Checks are missing from the checklist. The template defines these items which are absent from the PR body:
[ ] Plugin freshness: npm run plugin:generate[ ] Docusaurus tests: npm run docs:test
The PR description does mention under Testing that plugins were regenerated successfully, but the formal checklist items still need to be present and checked.
Coding Standards
scripts/collections/Modules/CollectionHelpers.psm1 — Split-CollectionMdByMarkers (line 620) is missing [OutputType([hashtable])]. All public module functions must carry an [OutputType] annotation per the repository's PowerShell standards (PSUseOutputTypeCorrectly). See inline comment.
Minor: Split-CollectionMdByMarkers uses the shorthand [Parameter(Mandatory)] while the surrounding codebase convention uses [Parameter(Mandatory = $true)]. Recommend aligning for consistency.
Code Quality
🔴 Markdown table injection (functional bug)
Both Prepare-Extension.ps1:461 and Generate-Plugins.ps1:361 write $entry.Description directly into a markdown table cell without escaping pipe (|) characters. If any artifact's description frontmatter field contains a |, the generated table will be malformed — columns shift and rendered output breaks. The fix is a one-line sanitize before the AppendLine call. See inline comments on both lines.
⚠️ Path-normalization regex in Generate-Plugins.ps1:331
The regex '^\.\//' (matches .//) is inconsistent with '^\./' used in Prepare-Extension.ps1:429 and never matches any current item path. This is a latent defect — see inline comment.
💡 Duplicated artifact table generation logic
The artifact table building block (group items by kind → sort → AppendLine per entry → write back via Set-ContentIfChanged) is implemented identically in both New-CollectionReadme (Prepare-Extension.ps1) and the #region Update collection.md artifact tables block (Generate-Plugins.ps1). Extracting a shared helper such as New-ArtifactTableMarkdown into CollectionHelpers.psm1 would eliminate this duplication and make the pipe-escaping fix a single change. Not a blocker, but a follow-up worth tracking.
Action Items
- ❌ Add the
## Security Considerationssection with its three checkboxes completed. - ❌ Add and check
Plugin freshness: npm run plugin:generateandDocusaurus tests: npm run docs:testin the Required Automated Checks section. - ❌ Add
[OutputType([hashtable])]toSplit-CollectionMdByMarkers. - ❌ Escape
|characters in$entry.Descriptionbefore writing to markdown table cells — applies to bothPrepare-Extension.ps1:461andGenerate-Plugins.ps1:361. ⚠️ Fix path regex inGenerate-Plugins.ps1:331from'^\.\//'to'^\./'.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none- refresh plugins/hve-core-all/README.md and plugins/security/README.md after collection.md autogen changes - pick up new security-review-sbd prompt and owasp-cicd, secure-by-design skills - update OWASP wording to security skill descriptions 🔒 - Generated by Copilot
There was a problem hiding this comment.
Automated Quality Review
This PR adds auto-generation of collection.md artifact tables from YAML manifests with channel-based maturity filtering and sentinel marker support. The implementation is well-structured, the testing is thorough (19 new tests covering maturity filtering, marker handling, and edge cases), and the encoding fix (utf8NoBOM) is a genuine correctness improvement. Two required fixes are needed before merge.
Issue Alignment
Issue #1256 could not be read directly (integrity policy), but the PR description matches a feature addition for auto-generated collection documentation with maturity filtering. Alignment appears correct based on the description.
PR Template Compliance
✅ Description is complete and detailed.
✅ Closes #1256 linked correctly.
✅ "New feature" and "Script/automation" checkboxes match the actual changes.
✅ Testing section documents automated results thoroughly.
✅ All checklist items checked appropriately.
Coding Standards Findings
1. [ValidateNotNullOrEmpty()] missing on Split-CollectionMdByMarkers.$Content — blocking
See inline comment on CollectionHelpers.psm1 line 623. The test 'Throws for empty string input' expects an exception, but [Parameter(Mandatory)] on [string] does not reject ''; only $null is rejected. Without [ValidateNotNullOrEmpty()] the test fails and the function silently returns a garbage result for empty input.
2. [OutputType([hashtable])] missing on Split-CollectionMdByMarkers — blocking
See inline comment on CollectionHelpers.psm1 line 620. The sibling Get-ArtifactDescription correctly declares [OutputType([string])]; this function must do the same per the repo's enforced PSUseOutputTypeCorrectly PSScriptAnalyzer rule. The .OUTPUTS doc-comment is not a substitute.
Code Quality Findings
3. Regex copy-paste inconsistency in Generate-Plugins.ps1 line 331 — non-blocking
See inline comment on Generate-Plugins.ps1 line 331. The pattern '^\.//' (two slashes) was copied from Prepare-Extension.ps1 but gained an extra /, making it '^\.//' instead of the correct '^\./. It has no runtime impact today because YAML paths don't use ./ prefixes, but the inconsistency will silently break if that convention ever changes.
4. Significant code duplication between Generate-Plugins.ps1 and New-CollectionReadme — suggestion
Lines 318–372 of Generate-Plugins.ps1 duplicate ~50 lines of the artifact-section-building loop that already exists in Prepare-Extension.ps1 (New-CollectionReadme, lines 410–464). The only meaningful difference is that Generate-Plugins.ps1 omits maturity filtering. Extracting a shared Build-ArtifactSections helper into CollectionHelpers.psm1 would eliminate drift risk and make future changes to table format or column layout a single-site edit.
Required Actions
- Add
[ValidateNotNullOrEmpty()]to the$Contentparameter ofSplit-CollectionMdByMarkers. - Add
[OutputType([hashtable])]toSplit-CollectionMdByMarkers. - (Optional but recommended) Fix regex in
Generate-Plugins.ps1line 331:'^\.\//' → '^\./'. - (Optional) Extract shared artifact-section builder to reduce duplication.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none- add [OutputType([hashtable])] and [ValidateNotNullOrEmpty()] to Split-CollectionMdByMarkers - fix regex typo '^\.\//' -> '^\./' in Generate-Plugins.ps1 resolvedPath computation - regenerate plugin READMEs 🔒 - Generated by Copilot
There was a problem hiding this comment.
Advisory review — this PR is from a maintainer. Findings are informational only.
Review Summary
The implementation is clean and the refactoring trajectory is solid. The three items flagged as blocking in the most recent automated CHANGES_REQUESTED review (#4141595773) — [ValidateNotNullOrEmpty()], [OutputType([hashtable])], and the regex inconsistency — are all confirmed resolved in the current branch (CollectionHelpers.psm1 lines 621–624; Generate-Plugins.ps1 line 331). The previous CHANGES_REQUESTED verdict no longer reflects the current code state.
Two template compliance gaps and two advisory code-quality observations remain.
Issue Alignment ✅
Closes #1256 is referenced. The PR delivers auto-generation of collection.md artifact tables with channel-based maturity filtering, sentinel-marker write-back, UTF-8 NoBOM encoding correction, and 19 new Pester tests. Scope is coherent. No regressions identified.
PR Template Compliance ⚠️
Two required sections from .github/PULL_REQUEST_TEMPLATE.md are absent from the PR body:
1. ## Security Considerations section is missing entirely.
The template requires three checkboxes under this heading:
[ ] This PR does not contain any sensitive or NDA information[ ] Any new dependencies have been reviewed for security issues[ ] Security-related scripts follow the principle of least privilege
2. Two Required Automated Checks are not listed in the checklist:
[ ] Plugin freshness: npm run plugin:generate[ ] Docusaurus tests: npm run docs:test
Both commands are defined in the template and must appear in the PR body. The Testing section narrative confirms plugins were regenerated successfully, but the formal checklist items still need to be present and ticked.
Previously Blocking Items — Now Resolved ✅
| Finding | Previous Review | Current State |
|---|---|---|
[ValidateNotNullOrEmpty()] missing on Split-CollectionMdByMarkers.$Content |
#4141595773 blocking | ✅ Added at line 624 |
[OutputType([hashtable])] missing on Split-CollectionMdByMarkers |
#4141595773 blocking | ✅ Added at line 621 |
Regex '^\.\//' (double-slash) in Generate-Plugins.ps1 |
#4141595773 non-blocking | ✅ Corrected to '^\./' at line 331 |
Note: the two open review threads for [ValidateNotNullOrEmpty()] and [OutputType] are not yet marked resolved on GitHub, but the implementation confirms they have been addressed.
Coding Standards ✅
All three previously flagged conventions issues are resolved. Split-CollectionMdByMarkers in CollectionHelpers.psm1 now has full comment-based help, [CmdletBinding()], [OutputType([hashtable])], and [ValidateNotNullOrEmpty()]. Get-ArtifactDescription is also well-formed.
Code Quality 💡
Artifact-table-building duplication — See inline comment on Generate-Plugins.ps1 lines 310–376. The added block replicates the item-iteration, StringBuilder table-generation, and marker write-back logic already present in New-CollectionReadme. Extracting a shared Build-CollectionArtifactSection helper into CollectionHelpers.psm1 would give a single place to evolve table format and improve test isolation. Not blocking, but worth tracking as a follow-on.
Test coverage for Generate-Plugins.ps1 new block — Codecov reports 6.66% patch coverage for the 68 lines added to Generate-Plugins.ps1 (42 lines uncovered). The marker-parsing logic is well-covered via unit tests on Split-CollectionMdByMarkers, but the integration path through Invoke-PluginGeneration with markers present is not exercised. A test scenario driving Invoke-PluginGeneration against a fixture with a marker-containing collection.md would close this gap.
Documentation 💡
docs/contributing/ai-artifacts-common.md and .github/copilot-instructions.md still have no mention of the two-zone collection.md convention (hand-authored intro vs. auto-generated artifact table), the marker format, or the command to sync the table after editing a YAML. This was raised by @katriendg in review #4074803431. A short ### Collection Markdown Auto-Generation subsection covering the marker format, the two-zone rule, and npm run plugin:generate as the sync command would prevent contributors from inadvertently editing the auto-generated zone.
Action Items
- Add the
## Security Considerationssection with all three checkboxes checked. - Add and check
npm run plugin:generateandnpm run docs:testin the Required Automated Checks list. - (Optional follow-on) Mark the two open review threads for
[ValidateNotNullOrEmpty()]and[OutputType]as resolved, since the fixes are in the code. - (Advisory) Track a follow-on issue for extracting
Build-CollectionArtifactSectionand addingdocs/contributingcoverage for the two-zone convention.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none… regenerate plugins)
There was a problem hiding this comment.
Review Summary
This PR adds auto-generation of collection.md artifact tables with channel-based maturity filtering. The core feature logic is solid — the marker-delimiter approach, Split-CollectionMdByMarkers, the encoding fix (utf8NoBOM), and the 19 new Pester tests are all well-implemented. However, there are PR template compliance gaps and code duplication issues that need to be addressed before merging.
Issue Alignment
Linked issue #1256 is referenced via Closes #1256. The PR description is detailed and the changes match the stated intent: marker-based write-back, channel filtering, and validation warnings for mismatched markers.
PR Template Compliance
❌ Security Considerations section is missing
The PR template requires a Security Considerations section with three checkboxes:
## Security Considerations
* [ ] This PR does not contain any sensitive or NDA information
* [ ] Any new dependencies have been reviewed for security issues
* [ ] Security-related scripts follow the principle of least privilegeThis section is entirely absent from the PR description. Since this PR modifies automation scripts that write files in-place (Set-ContentIfChanged), the third checkbox is directly relevant. Please add the section with checkboxes completed.
❌ Two required automated checks are missing from the checklist
The template's Required Automated Checks section includes entries for npm run plugin:generate and npm run docs:test that are not present in the PR's checklist:
[ ] Plugin freshness: npm run plugin:generate[ ] Docusaurus tests: npm run docs:test
The testing section narrative confirms plugin:generate ran successfully (14 plugins regenerated), but these must also appear as checked items in the Required Automated Checks checklist per template requirements.
Coding Standards
⚠️ Code duplication (see inline comment on Generate-Plugins.ps1 lines 310–376)
The artifact table generation block in Generate-Plugins.ps1 is a near-verbatim copy (~65 lines) of the same logic in New-CollectionReadme. This should be extracted into a shared CollectionHelpers.psm1 function.
⚠️ Duplicate channel-mapping functions (see inline comment on Prepare-Extension.ps1 lines 503–527)
Get-AllowedMaturities in Prepare-Extension.ps1 and Get-AllowedCollectionMaturities in Generate-Plugins.ps1 are functionally identical. The shared implementation belongs in CollectionHelpers.psm1.
⚠️ New-CollectionReadme missing [OutputType()] and leaking output (see inline comment on Prepare-Extension.ps1 lines 333–361)
Every other function in Prepare-Extension.ps1 declares [OutputType()]. This function also has an unassigned Set-ContentIfChanged call whose [bool] return value leaks onto the output stream.
Code Quality
💡 Design note — channel defaults write to the same source files
Generate-Plugins.ps1 defaults to Channel = 'PreRelease' when writing collection.md artifact tables, while Prepare-Extension.ps1 defaults to Channel = 'Stable'. Both write to the same *.collection.md files. The committed files in this PR correctly reflect the npm run plugin:generate PreRelease output. However, running Prepare-Extension.ps1 on the Stable channel would silently overwrite those files with a subset of items (stable-only), creating an inconsistent state. Consider documenting which script is the authoritative writer for collection.md, or making the channel default consistent between the two scripts.
Action Items
- Add the Security Considerations section to the PR description with appropriate boxes checked.
- Add
plugin:generateanddocs:testto the Required Automated Checks checklist in the PR description. - Extract the duplicated artifact table generation logic into a shared
CollectionHelpers.psm1function. - Consolidate
Get-AllowedMaturities/Get-AllowedCollectionMaturitiesinto a single exported function inCollectionHelpers.psm1. - Add
[OutputType([void])]toNew-CollectionReadmeand suppress theSet-ContentIfChangedreturn value with$null =.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | noneThere was a problem hiding this comment.
PR Review — feat/collection-md-autogen-maturity-1256
Overview: The PR successfully delivers the sentinel-marker–based auto-generation of collection.md artifact tables with channel-based maturity filtering. The design is coherent, the utf8NoBOM encoding fix is a genuine improvement, and Split-CollectionMdByMarkers + Set-ContentIfChanged are clean, reusable additions to CollectionHelpers.psm1. Two blocking issues need to be addressed before merge.
Issue Alignment
Could not read issue #1256 in full detail, but the PR description ("Closes #1256") and the implementation (marker support, Get-AllowedMaturities filtering, regenerated collection.md/plugin README.md artifacts) are internally consistent. No scope-creep concerns. ✅
PR Template Compliance
All sections filled. Type of change checkboxes are accurate. Testing section documents new test additions. Checklist is complete. ✅
Coding Standards
[System.Text.StringBuilder], foreach ($section in @(...)), | Name | Description | table, sorted entry loop) is copied verbatim from New-CollectionReadme into Generate-Plugins.ps1 lines 346–364. Now that CollectionHelpers.psm1 exists as a shared module, this logic belongs there. See inline comment.
Minor: Get-AllowedCollectionMaturities in Generate-Plugins.ps1 (line 76) and Get-AllowedMaturities in Prepare-Extension.ps1 (line 503) return identical arrays for the same channel input. With CollectionHelpers.psm1 as the natural home, there is an opportunity to consolidate these into one exported function. Non-blocking but worth a follow-up.
Minor: The .PARAMETER Channel help text on Invoke-ExtensionCollectionsGeneration describes the parameter purpose but does not list valid values (Stable, PreRelease), unlike the top-level script parameter block which does enumerate them.
Code Quality
🔴 Blocking — Missing test coverage in Generate-Plugins.ps1 (Codecov: 6.66% patch, 42 lines uncovered): The entire #region Update collection.md artifact tables block is exercised by no existing test. The 19 new tests target New-CollectionReadme and Split-CollectionMdByMarkers exclusively; Generate-Plugins.ps1 has no dedicated test coverage for this new behaviour. The DryRun guard, the happy-path write-back, the marker-absent fallthrough, and the empty-collection case are all untested. See inline comment.
💡 Design observation — Footer positioning in README template: When a collection.md has a footer, it is appended to $bodyForTemplate and rendered into {{BODY}}, which appears before ## Included Artifacts in the README template. Footer content (section headings, notes) would therefore render before the artifact tables. See inline comment. No current collection has a non-empty footer, so no regression today, but the contract is surprising.
Action Items
- Add tests for the
Generate-Plugins.ps1collection.md update block — happy path,DryRunguard, no-markers fallthrough, empty items. (Blocking) - Extract the duplicated artifact table builder into a shared
CollectionHelpers.psm1function and call it from bothGenerate-Plugins.ps1andNew-CollectionReadme. (Blocking — resolves duplication and will naturally bring coverage through the shared module tests) - Document or fix the footer-in-
{{BODY}}positioning for future collection authors. (Non-blocking follow-up)
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none- add synth-data-generate to data science README - include customer-card-render in experimental README - document dt-canonical-deck in hve-core-all README 🔧 - Generated by Copilot Signed-off-by: Marcel Bindseil <[email protected]>
There was a problem hiding this comment.
Advisory review — this PR is from a maintainer. Findings are informational only.
Review Summary
The PR successfully delivers auto-generation of collection.md artifact tables with channel-based maturity filtering and sentinel marker support. The description is clear, the linked issue is present, type-of-change selections accurately reflect the changes, and the 19 new Pester tests provide solid coverage. All prior blocking review threads are marked resolved. The two human reviewers (katriendg, rezatnoMsirhC) have approved.
Issue Alignment
✅ The changes align with the stated goal (auto-generation of collection artifact tables with maturity filtering). The collection.md writeback was correctly added to Generate-Plugins.ps1 so npm run plugin:generate keeps both plugins/*/README.md and collections/*.collection.md in sync.
PR Template Compliance
Two gaps relative to the current .github/PULL_REQUEST_TEMPLATE.md:
[ ] This PR does not contain any sensitive or NDA information[ ] Any new dependencies have been reviewed for security issues[ ] Security-related scripts follow the principle of least privilege
None of these appear in the PR description. For completeness, these should be filled in before merge.
npm run plugin:generate— plugin generation results are described in the Testing narrative but the checklist item is absentnpm run docs:test— no mention of this check in either the checklist or the Testing section
The template now requires both items to be explicitly checked. The author may wish to update the checklist to reflect that plugin:generate was run (evidenced by "All 14 plugins regenerated successfully") and confirm the outcome of docs:test.
Coding Standards
✅ Split-CollectionMdByMarkers in CollectionHelpers.psm1 correctly includes [CmdletBinding()], [OutputType([hashtable])], [ValidateNotNullOrEmpty()], and full comment-based help — a prior blocking violation that has been resolved.
✅ Marker constants ($CollectionMdBeginMarker, $CollectionMdEndMarker) are now centralized in CollectionHelpers.psm1 as exported module variables — another prior violation that has been resolved.
Code Quality
The following observations were raised in prior review threads (all now marked resolved). They are restated here for completeness, as the code changes have not yet landed:
💡 Duplicate artifact table-building logic — The #region Update collection.md artifact tables block in Generate-Plugins.ps1 (lines ~310–376) and the corresponding section in New-CollectionReadme in Prepare-Extension.ps1 implement materially identical StringBuilder + foreach ($section in @(...)) logic. Extracting a shared Build-CollectionArtifactSection helper into CollectionHelpers.psm1 would reduce future table-format drift risk. Non-blocking; the prior resolved thread noted this as a refactoring opportunity.
💡 Footer placement in New-CollectionReadme — When $parsed.HasMarkers is true, $parsed.Footer is concatenated into $bodyForTemplate, which maps to {{BODY}} in the template. Because {{BODY}} precedes {{ARTIFACTS}} in the template order, any future footer content (e.g., ## Prerequisites) will render before the artifact table in the generated README rather than after it. No current collection.md file has a non-empty footer, so there is no visible regression today. A {{FOOTER}} token at the end of the template would make the document order explicit and predictable for future collection authors.
💡 Missing test coverage for Generate-Plugins.ps1 writeback path — The #region Update collection.md artifact tables block has no dedicated Pester tests (no happy-path test, no -DryRun guard test, no empty-items test). The prior review thread flagged this; it remains unaddressed. This does not block the merge but creates a coverage gap for a non-trivial code path.
Action Items
- Add the Security Considerations section with appropriate checkbox attestations.
- Add
plugin:generateanddocs:testto the Required Automated Checks checklist (or update the PR description to confirm both passed). - (Optional follow-up) Extract shared artifact table builder and add
Generate-Plugins.ps1writeback tests in a subsequent PR.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1256
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none## Pre-Release 3.3.101 ### ✨ Features - add removed maturity tier and retire owasp-docker (#1444) - add evaluation dataset creator (#1279) - align RAI planner with guide, remove scoring, improve UX (#1287) - add PSGallery staleness check and BOM cleanup (#1379) - ISA-95 network planner agent (#1177) - auto-generate collection.md with maturity filtering (#1316) - add folder-consistency check and standardize WARN outp… (#1350) - add synth-data-generate prompt to data-science collection (#1419) - add canonical deck workflow and customer-card rendering for design thinking (#1413) - add Figma MCP integration for DT artifact export (#1222) - introduce `owasp-docker` (#1245) - replace hve-core-specific references with portable discovery-based language (#1335) - introduce `owasp-cicd` (#1246) - add secure-by-design knowledge skill (#1223) - introduce `owasp-infrastructure` (#1244) - introduce `owasp-mcp` (#1207) - add OutputPath parameter to Invoke-LinkLanguageCheck.ps1 (#1229) - add -OutputPath parameter to Validate-SkillStructure.ps1 (#1225) - add maintainer-only skip-review label guard (#1293) - add extension collections overview and integrate into getting started flow (#950) - add agentic workflows for automated issue triage, implementation, PR review, dependency review, and doc-staleness detection (#1219) - consolidate package-lock.json version sync into Update-VersionFiles.ps1 (#1240) - add standards code review agent and full review orchestrator (#1174) - standardize pytest-mock as Python mocking framework (#1170) - add Jira backlog workflows and Jira/GitLab skills (#978) - add centralized version bump script and supply-chain attestation (#1183) ### 🐛 Bug Fixes - pin PowerShell-Yaml to 0.4.7 across all install sites (#1378) - close fork-PR/workflow-file-PR secret-strip gap and normalize upload-artifact version (#1421) - replace stream-based lookahead with array indexing in list-changed-files.sh (#1376) - centralize ISO 8601 timestamp regex in CIHelpers (#1343) - update stale documentation date in release-process.md (#1363) - pin basic-ftp to 5.3.0 to resolve GHSA-rp42-5vxx-qpwr (#1374) - add bot filter to dependency PR review workflow (#1362) - resolve pip-audit findings in powerpoint, gitlab, and jira skill lock files (#1360) - standardize Timestamp JSON key casing across all lint result files (#1314) - add synchronize trigger to PR Review workflow (#1323) - standardize timestamp in Validate-SkillStructure.ps1 to use Get-StandardTimestamp (#1280) - add parallel subagent dispatch and structured JSON contracts to code-review-full (#1304) - standardize timestamp in SecurityHelpers.psm1 to use Get-StandardTimestamp (#1284) - standardize timestamps in Test-DependencyPinning.ps1 and SecurityClasses.psm1 (#1282) - derive collection artifact counts from YAML at build time (#1275) - standardize timestamp in FrontmatterValidation.psm1 to use Get-StandardTimestamp (#1285) - standardize timestamp in Markdown-Link-Check.ps1 to use Get-StandardTimestamp (#1283) - escape hyphens in Mermaid diagram on Collections page (#1262) - add summary timestamp to PSScriptAnalyzer output (#1211) - fix plugin compatibility and robustness for coding-standards code review agents (#1289) - standardize timestamp in Test-CopyrightHeaders.ps1 to use Get-StandardTimestamp (#1278) - standardize timestamp in Invoke-YamlLint.ps1 to use Get-StandardTimestamp (#1270) - standardize timestamp in Invoke-LinkLanguageCheck.ps1 to use Get-StandardTimestamp (#1264) - fix dependency-review path filters and sparse-checkout cone mode (#1259) - replace invalid bare tool names with official tool identifiers (#1198) - fix broken links and remove orphaned reference in code review docs (#1257) - exclude Python env dirs from skill validation warnings (#1255) - pin happy-dom and serialize-javascript to resolve Dependabot vulnerabilities (#1253) - remove Mermaid diagram and add missing collection cards (#1247) - disable MCP servers by default to prevent token limit errors (#1144) - sync package-lock.json after pre-release version bump (#1236) - separate mermaid node declarations and add dynamic diagram generation with tests (#1215) - replace anchor links in meeting-analyst with bold text references (#1201) - remove recursive symlinks in jira and gitlab skill directories (#1233) - validate-installation scripts now check .github/skills directory (#1010) (#1206) - resolve npm audit vulnerabilities via dependency overrides (#1200) - add post-release triggers to scorecard workflow (#1186) - add missing .md extensions to relative links in agent documentation (#1180) ### 📚 Documentation - broaden Security Review description beyond OWASP (#1385) - document maintainer advisory mode and skip-review label guard (#1386) - document ExcludePaths/OutputPath for Invoke-LinkLanguageCheck (#1383) - CLI getting-started: clarify plugin install commands as alternatives (-all vs base) (#1251) ### ♻️ Refactoring - align agent and prompt folder names to collection identifier (#1210) ### 🔧 Maintenance - pin PSScriptAnalyzer to 1.25.0 and sync stale workflow version comments (#1389) - bump lxml from 6.0.2 to 6.1.0 in /.github/skills/experimental/powerpoint (#1424) - bump @vscode/vsce from 3.7.1 to 3.9.1 in the npm-dependencies group (#1390) - bump the github-actions group across 1 directory with 7 updates (#1391) - bump follow-redirects from 1.15.11 to 1.16.0 in /docs/docusaurus (#1356) - upgrade Node.js from 20 to 24 and bump cspell to v10 (#1353) - bump basic-ftp from 5.2.0 to 5.2.1 (#1324) - update github/gh-aw-actions requirement to 536ea1bad8c6715d098a9dc1afea8d403733acfe in the github-actions group across 1 directory (#1298) - update security instruction attributions and compliance (#1294) - bump the npm-dependencies group with 2 updates (#1297) - pre-release 3.3.41 (#1252) - streamline RAI Planner phase structure and documentation (#1273) - bump happy-dom from 20.8.8 to 20.8.9 in /docs/docusaurus (#1237) - pre-release 3.3.27 (#1191) - bump pygments from 2.19.2 to 2.20.0 in /.github/skills/gitlab/gitlab (#1234) - bump path-to-regexp from 0.1.12 to 0.1.13 in /docs/docusaurus (#1226) - bump the github-actions group with 4 updates (#1231) - add missing folders and alphabetize location lists (#1193) - bump brace-expansion (#1224) - bump handlebars from 4.7.8 to 4.7.9 in /docs/docusaurus (#1217) - bump brace-expansion from 5.0.3 to 5.0.5 in /docs/docusaurus (#1213) - pre-release 3.3.10 (#1187) - bump markdownlint-cli2 from 0.21.0 to 0.22.0 in the npm-dependencies group (#1175) - bump the github-actions group with 3 updates (#1176) - pre-release 3.3.1 (#1165) --- *Managed automatically by pre-release workflow.* Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Description
Added auto-generation of
collection.mdartifact tables from YAML manifests with channel-based maturity filtering and sentinel marker support. The generation engine in Prepare-Extension.ps1 gained a-Channelparameter that controls which maturity levels appear in output, andNew-CollectionReadmewas rewritten to parse marker-delimited sections, filter items by maturity, and write back updated content preserving hand-written introductions and footers. Collection validation now detects mismatched or reversed markers as warnings for backward compatibility. The encoding forSet-ContentIfChangedwas corrected to UTF-8 without BOM. Nineteen new Pester tests cover maturity filtering, template marker handling, andSplit-CollectionMdByMarkersedge cases.Related Issue(s)
Closes #1256
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md)Other:
.ps1,.sh,.py)Sample Prompts (for AI Artifact Contributions)
Testing
Automated testing:
Split-CollectionMdByMarkers: no markers, empty string throws, correct intro/footer parsing, partial markers, reversed markers, duplicate markersManual testing: Not performed.
Checklist
Required Checks
AI Artifact Contributions
Required Automated Checks
The following validation commands must pass before merging:
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-linksnpm run lint:ps