Skip to content

Declaration scaffolds hand users blanks for values shipgate already computed (root binding candidate, accepted_values) #388

Closed

Description

@pengfei-threemoonslab

Impact

suggested-declarations.yaml is the file the tool tells users to edit. It hands them
<REVIEW_REQUIRED> blanks for values shipgate has already computed and stored elsewhere.
The user is asked to guess, or to go spelunking in a 242 KB report.json, for answers the
tool already has.

Two concrete instances, both on the first-adoption path.

Severity: P2 — the dominant cost of first adoption

Neither is a correctness bug; together they are why a first scan lands in
insufficient_evidence and stays there. Adoption friction is the constraint on this product
far more than engine accuracy.

Instance 1 — init computes the root agent, then writes a manifest without it

agents-shipgate init --workspace . --write --json

Reports, with high confidence:

"agent_name": "SmartCloserAgent",
"agent_name_candidates": [{
  "value": "SmartCloserAgent", "role": "root_agent",
  "rank_score": 4.0, "selectable": true,
  "rationale": ["declared as Agent(name=…) in agent.py",
                "bound as App(root_agent=root_agent)"]
}]

The manifest it writes contains agent.name and no agent_bindings.root. The very next
scan returns:

insufficient_evidence: the root agent is ambiguous (agent_binding_graph).
Fix at shipgate.yaml#agent_bindings.root

The scaffold then asks for two <REVIEW_REQUIRED> values — source_id and object — whose
correct answers (adk_agent, root_agent) init derived one command earlier and printed in
its own rationale.

This is not a request to auto-declare the trust root. Inferring agent_bindings.root
from AST evidence is the self-declaration attack surface from #268, and refusing to guess is
correct. The ask is that the scaffold offer the computed candidate for a human to confirm:

# closes: ambiguous_root_agent
# merge into: shipgate.yaml#agent_bindings.root
# shipgate detected: source_id: adk_agent, object: root_agent
#   (declared as Agent(name=…) in agent.py; bound as App(root_agent=root_agent))
#   Confirm these are correct, then replace the placeholders below.
agent_bindings:
  root:
    source_id: <REVIEW_REQUIRED>
    object: <REVIEW_REQUIRED>

The human declaration property is preserved. The guessing game is removed.

Instance 2 — the scaffold omits accepted_values that report.json already carries

suggested-declarations.yaml emits:

effect: <REVIEW_REQUIRED>
authority:
  mode: <REVIEW_REQUIRED>

with no vocabulary. agents-shipgate explain takes only a check_id, so there is no CLI
route to the answer.

Meanwhile report.json carries it exactly, per gap, at
release_decision.evidence_coverage.evidence_gaps[].next_action.accepted_values:

"accepted_values": ["read", "write", "destructive",
                    "external_communication", "financial_write", "production_operation"]
"accepted_values": ["none", "scoped", "unscoped", "ambient"]

The data exists and is per-field accurate. It is simply not written into the file users are
directed to fill in. (The scaffold header does document the authority.mode co-requirement
rules, so the omission is inconsistent rather than total.)

Expected behavior

The scaffold is self-sufficient: for every <REVIEW_REQUIRED>, it carries the allowed values
and any candidate shipgate computed, as comments.

Acceptance criteria

  • Every <REVIEW_REQUIRED> in suggested-declarations.yaml is preceded by a comment
    listing that field's accepted_values.
  • Where a candidate was computed (root agent object/source), it appears as a commented
    suggestion with its rationale, still requiring human confirmation.
  • A user can complete the scaffold without opening report.json or the JSON schema.
  • Test asserting scaffold accepted_values comments stay in sync with the gap's
    accepted_values (so the vocabulary cannot drift between the two artifacts).

Notes

Found while running the full adoption path on
google/adk-samples#1745 as a first-time
user, following only CLI output. These two items plus #385 and #386 were the reasons that run
never reached a gate-able verdict.

Activity

  1. pengfei-threemoonslab commented on Aug 20, 2026

    @pengfei-threemoonslab
    ContributorAuthor

    Re-checked on 0.16.0b7 (contract 24) during a first-time adoption walk of google/adk-samples#1745. Half of this is fixed; the accepted_values half is not.

    Fixed — the root binding candidate. Scoped init --workspace python/agents/smart_closer --write now derives and writes agent.name: SmartCloserAgent instead of leaving CHANGE_ME, with the rationale chain visible:

    "rationale": [
      "declared as Agent(name=…) in agent.py",
      "resolved from the static default of an environment lookup in config.py — overridable at runtime",
      "bound as App(root_agent=root_agent)"
    ]

    placeholders[] is down to a single entry (agent.declared_purpose[0]).

    Still open — the scaffold never names the accepted values. suggested-declarations.yaml has improved a lot (a header explaining co-required authority fields, and # closes: / # merge into: comments per block), but it still hands out bare blanks:

    # closes: missing_authority_evidence, missing_effect_evidence
    # merge into: shipgate.yaml#action_surface.actions[tool='create_salesforce_quote']
    tool: create_salesforce_quote
    scopes:
    - <REVIEW_REQUIRED>
    authority:
      mode: <REVIEW_REQUIRED>
      auth_type: <REVIEW_REQUIRED>
      reason: <REVIEW_REQUIRED>
    effect: <REVIEW_REQUIRED>

    Measured on the generated file:

    $ grep -c "^tool:" suggested-declarations.yaml                                  # 12
    $ grep -c "financial_write\|external_communication\|destructive" ...yaml        # 0
    $ grep -c "unscoped\|ambient" ...yaml                                           # 0
    

    Meanwhile report.json carries both vocabularies in every gap's next_action.accepted_values — the nine effect values and the four authority modes. So the file the user is told to edit is the one file that does not say what a legal answer looks like.

    Concrete adoption cost on this PR: 12 blocks × 4 fields ≈ 48 hand-entered values, with the legal vocabulary in a different file, before a gate verdict is reachable.

    Suggestion unchanged from the original report: emit accepted_values as a comment above each blank, e.g. effect: <REVIEW_REQUIRED> # read | write | destructive | external_communication | financial_write | production_operation | privileged_data_access | code_execution | identity_access.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions