Skip to content

fix(security): route tracker imports through vetted issue creation - #1982

Merged
potiuk merged 3 commits into
apache:mainfrom
KatalKavya96:fix/1979-security-import-vetted-op
Nov 4, 2026
Merged

potiuk merged 3 commits into
apache:mainfrom
KatalKavya96:fix/1979-security-import-vetted-op

Conversation

@KatalKavya96

Copy link
Copy Markdown
Contributor

Summary

  • Fixes security-issue-import: create the tracker via vetted-op issue-create, not raw gh api (agent-guard blocks it) #1979 by replacing raw gh api tracker creation with policy-scoped issue-create-labeled across all three security import skills, preventing agent-guard blocks on confidential reports.
  • Adds atomic label validation, private-tracker targeting, and node_id support while preserving the existing issue-create contract and non-GitHub backends.
  • Updates caller permissions, setup guidance, bundled vetted-ops implementation, and regression tests for security and failure handling.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
    (PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
    (a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
  • Other:
    • SKIP=workspace-pytest prek run --all-files passed (exit code 0).
    • Full vetted-ops pytest suite passed; 13 native Codex evaluator tests skipped because the evaluator was unavailable.
    • Regression tests cover approved labels, private-tracker targeting, confidential body transmission through stdin, and command failures.
    • Bundled vetted-ops implementation verified against canonical source.
    • No real security tracker issues were created during testing.

RFC-AI-0004 compliance

  • HITL — existing explicit user confirmation remains required before tracker creation.
  • Sandbox — tracker writes use the existing policy-scoped vetted-ops dispatcher; no unrestricted host access added.
  • Vendor neutrality — existing tracker placeholders and non-GitHub backend support are preserved.
  • Conversational + correctable — setup guidance documents approval of missing policy grants and labels.
  • Write-access discipline — creation requires an authorized caller, approved labels, and the existing confirmation workflow.
  • Privacy LLM — confidential report bodies remain in the existing handling workflow and are transmitted to GitHub through stdin rather than command arguments.

Linked issues

Fixes #1979

Notes for reviewers (optional)

  • Added issue-create-labeled rather than changing the existing issue-create signature to preserve compatibility.
  • Labels are validated against the configured allowlist and applied atomically. Missing permissions or labels fail closed; setup proposes changes for explicit user approval.
  • GitHub-specific behavior is explicitly unavailable on Jira and local-files backends, which continue using issue-create.
  • Please review the policy-validation boundary and the three updated import workflows.

@github-actions github-actions Bot added family:security security-* skills family:setup setup-* skills family:tools tools/* family:docs Docs, MISSION.md, READMEs capability:intake Import external signal + sync tracker state capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) substrate:sandbox Tool substrate: agent isolation, egress control, settings audit labels Nov 4, 2026
@KatalKavya96

KatalKavya96 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

@potiuk , Implemented the fix for #1979.

  • Tracker creation now uses the policy-scoped issue-create-labeled operation across all three security import workflows, with atomic label validation and confidential body handling through stdin.

  • Added regression tests, updated caller grants and setup guidance, and preserved the existing issue-create contract for non-GitHub backends.

Validation: Full vetted-ops test suite passed, along with Ruff, mypy, and all enabled repository hooks.

Ready for review. Thanks!

@github-actions github-actions Bot added the substrate:framework-dev Tool substrate: build / validate / eval the framework itself label Nov 4, 2026

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direction is right and the label and tracker handling hold up, but the documented $(cat …) command cannot run under the secure setup, so every GitHub import would stop at creation until the title reaches vetted-op another way.

Golden rule — launch vetted operations with the literal ~ path (plugins/magpie-security/skills/issue-import/apply.md:52)

The same holds for the eval wrapper, ~/.claude/scripts/magpie-run-evals.sh, and for any sandbox-excluded command:
write it bare, with the literal ~, as the whole command line (not behind env or command), or it runs inside the sandbox and fails.

"$(cat <scratch>/issue-title-<threadId>.txt)" \

The $(cat …) keeps the command from matching the sandbox exclusion, so under the secure setup vetted-op runs inside the sandbox and fails before it reaches GitHub. I confirmed this with a read-only vetted-op-read --caller "$(echo …)" viewer, which dies with PermissionError: Operation not permitted while loading the policy; the same command without the substitution works. Since this step says to stop on failure and never fall back to raw gh, every GitHub import stops here. The same applies to issue-import-from-md/SKILL.md:362 and issue-import-from-pr/SKILL.md:470 and :472. Inlining the title is not an option, because it is attacker-controlled. Let issue-create-labeled take the title as a file in the workspace, validated like the body and passed to gh as -F title=@<file>, so the command needs no substitution. The labels can be written inline as a literal JSON argument, since they come from configuration rather than from the report.

Smaller observations

  • tools/vetted-ops/README.md:618 — The table below now has eleven rows. The same text is in the plugins/magpie-vetted-ops copy. Update the count, or move the GitHub-only row out of this table.
  • general — The eval box is unticked and the test plan names no eval run, while the PR changes the security-issue-import, -from-md and -from-pr skills and adds setup fixtures. Run those three suites and the two setup suites, and record the result in the PR.
  • See inline comments on tools/vetted-ops/src/vetted_ops/ops.py:1107.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

Comment thread tools/vetted-ops/src/vetted_ops/ops.py Resolved
@potiuk
potiuk force-pushed the fix/1979-security-import-vetted-op branch from ffe1fb5 to c93e998 Compare November 4, 2026 13:38
The documented "$(cat <title file>)" kept vetted-op out of the sandbox
exclusion, so every GitHub import failed under the secure setup. The
title is now a workspace file the dispatcher reads and validates, like
the body, so the command stays bare and the shell never sees it.

A dry run accepts any --config, so it never prints a title read from
a file, and a refused title file is not quoted in the error.

Also corrects the operation count in the vetted-ops README.

Generated-by: Claude Opus 5
@potiuk
potiuk force-pushed the fix/1979-security-import-vetted-op branch from c93e998 to 182a872 Compare November 4, 2026 13:44

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I applied the changes myself rather than asking for another round, because I want to bump the dev version with this fix in it, and I rebased the branch onto main. Please pull before pushing again.

What changed in 182a872:

  • issue-create-labeled now takes the title as a workspace file, read and validated by the dispatcher like the body (tools/vetted-ops/src/vetted_ops/ops.py, cli.py), so the commands in issue-import/apply.md, issue-import-from-md/SKILL.md and issue-import-from-pr/SKILL.md have no $(cat …) and run under the secure setup. The labels are written inline.
  • A dry run never prints a title read from a file, and a refused title file is not quoted in the error, because a dry run accepts any --config.
  • The README now says eleven operations.

The title still reaches gh as an argument inside the dispatcher, the same way issue-create does it. It never reaches a shell, so I left that as it is.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Magpie handles maintainer review:
Contributing guide.

@potiuk
potiuk merged commit c748ccd into apache:main Nov 4, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:intake Import external signal + sync tracker state capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) family:docs Docs, MISSION.md, READMEs family:security security-* skills family:setup setup-* skills family:tools tools/* substrate:framework-dev Tool substrate: build / validate / eval the framework itself substrate:sandbox Tool substrate: agent isolation, egress control, settings audit

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security-issue-import: create the tracker via vetted-op issue-create, not raw gh api (agent-guard blocks it)

2 participants