RFC: evidence-first declarations — ask only what the scanner cannot prove, pin every answer to its evidence #410

Problem

The declaration workflow is the last wall between a first-time user and a gateable verdict, and it is currently shaped wrong in four measured ways (all from the 2026-08-20/23 adoption walks, CLI 0.16.0b7, post-#403):

1. It asks humans to re-state what the scanner already knows. smart_closer: 12 tools × 4 fields ≈ 48 hand-typed values, while the same report already carries derived risk_tags for the interesting tools. The user's question "what should I write?" has no answer in the product; the vocabulary's semantics (ambient vs unscoped) appear in no glossary.
2. It accepts contradictions silently — A declaration weaker than inferred evidence is accepted silently: effect: read on a tool tagged external_write becomes pass-eligible with zero findings #409. The declarant is the person the gate is blocking; the path of least resistance is read/none, and shipgate converts it into pass-eligibility with zero findings.
3. It flattens three epistemically different statements into one form. effect is a fact about code (derivable here). authority/scopes are facts about deployment (they live in Terraform/OAuth config, often unanswerable from the repo — both walk targets are samples whose honest authority is none, which trivially passes). approval is an organizational rule. Different owners, different sources of truth, different rot rates — one undifferentiated blank list.
4. Nothing ever re-opens a declaration. Matching is by name (tool_id optional, schemas/manifest/action_surface.py:150); a year-old effect: write keeps passing after the function's semantics change. Declarations rot, and a green gate at month twelve rests on descriptions of functions that no longer do what they did.

And for the coding-agent workflow specifically, the current model creates a pincer (hit live on github-mcp-server#3076): the agent must not edit shipgate.yaml, but its new tool is invisible until someone does.

Principles

1. Never ask a human what the scanner can prove. Strong evidence establishes an effect with no declaration; questions exist only where evidence is ambiguous, absent, or being overridden.
2. An agent may propose what evidence supports; only a human may assert against it. A content rule, enforced by comparison against derived evidence — no authorship forensics (git can't provide them; agents commit as their user).
3. Every answer is pinned to the evidence that justified it. Evidence moves → the question re-opens. One comparator serves contradiction checking (A declaration weaker than inferred evidence is accepted silently: effect: read on a tool tagged external_write becomes pass-eligible with zero findings #409) and drift detection.
4. Attestation is the PR review of a protected file, not a separate ceremony. The manifest stays the trust root; CODEOWNERS + branch protection supply the human-authority property in practice, and shipgate should say so.
5. Authority follows credentials, not functions. Declared once per source, overridden per tool.
6. Facts in the manifest; rules in policy packs. Chosen once at init, not answered 12 times.

Design

A. Monotone declarations (#409 — Increment 1)

Declarations may freely add or escalate relative to evidence. De-escalation requires an explicit override: {evidence, reason}; without it, config_error. With it, accepted — but always emits a review finding; an override is never silent. Checks consume the union of declared effect and derived risk evidence, so under-declaring a secondary effect cannot dodge a check (this also resolves multi-effect tools without breaking the effect enum: risk_tags already carries the rest).

B. Evidence-first effects + questionnaire scaffold

Adapters emit per-tool effect evidence with basis and a derivation_id. Strong basis (call-graph to a known SDK write; first-party MCP readOnlyHint/destructiveHint on a trust: internal source) → effect established, no blank. suggested-declarations.yaml becomes a questionnaire holding only the ambiguous residue, each entry a question with candidates, the evidence for each, and a proposed conservative answer:

# ── Question 2 of 5 ──────────────────────────────────────────
# get_sap_credit_status: evidence is ambiguous between two readings.
#   read                   — returns data, no state change observed
#   privileged_data_access — the data is a customer credit limit
- tool: get_sap_credit_status
  effect: privileged_data_access   # proposed; keep to confirm, or replace
  basis: confirmed:d41f8a2c        # evidence pin, stamped on apply

Verify reports progress as "3 of 5 questions answered; 2 remain (both authority)" instead of "36 semantic evidence gaps" — same facts, visible finish line.

Measured cost: smart_closer 48 blanks → ~5 questions + 1–2 authority blocks. github-mcp-server (117 tools, first-party MCP annotations) → 1 authority block + confirmations for the destructive handful.

C. Per-source authority

tool_sources:
  - id: adk_agent
    type: google_adk
    path: agent.py
    authority: {mode: scoped, auth_type: oauth2, credential_mode: service_account}

All six Salesforce tools share one client; asking per-tool asks the same infra question six times and breeds the copy-paste that breeds wrong answers. Per-tool override remains; existing per-tool declarations keep working (additive).

D. The coding-agent loop

One new next_action.kind: confirm_declarations carrying the question list, each tagged authorable_by: coding_agent | human:

• Agent applies what it may: evidence-consistent proposals via the existing apply-patches surface (new patch kind, --confidence high), committed to the PR branch — review_publishable already permits commit/push/update_pr.
• Human review happens at the PR, where it already happens. trust_root_touched still requires a human merge; the PR comment renders the declaration diff as the review surface — ✓ rows machine-verified as evidence-consistent, ⚠ rows for overrides. The reviewer reads exceptions, not 117 rows; that is what makes attestation real rather than rubber-stamped.
• Agent stops at authorable_by: human with the exact questions, not a generic "human review required."

This dissolves the #3076 pincer as a consequence: an agent declaring its own new tool is an evidence-consistent edit (fine, human merges); an agent weakening a declaration is blocked by content rule A regardless of what file it edits. Extends the existing quickstart doctrine ("proposal authorship, not approval") from tool_sources rows to declarations.

E. Drift pinning

basis: confirmed:<derivation_id> stamped at apply time; every scan re-derives and compares. Hash stable (renames, refactors) → silence, no treadmill. Hash moved → declaration_drift gap re-opens naming the delta ("send_email: external_communication evidence added since confirmation"). Same comparator as A. This is what makes the system safe to run for years — today nothing re-opens a declaration.

F. Policy packs as the rules layer

init asks one question: pick a pack (default, financial-strict, read-only-agent, template). Packs map effect → required controls. Today's seven separate SHIP-POLICY-APPROVAL-MISSING blockers become one statement: "pack default requires approval for financial writes; 3 tools lack it."

G. Adoption ladder + trust hardening

• Rung 0 is the missing consumer mode: both walk targets are samples someone is evaluating, for which producer ceremony is unanswerable in principle. A manifest-less catalog is also the 60-second demo.
• environment.target: template: sample repos adopt honestly — authority questions suppressed, verdict capped at review_required ("adopters must declare authority before production"). Prerequisite for the upstream-PR strategy (benchmark/upstream-prs.md) to be honest.
• SHIP-TRUST-MANIFEST-UNPROTECTED (static, guidance-level): no CODEOWNERS rule covers the manifest. Branch protection is not statically verifiable; the finding says so instead of pretending.
• doctor prints the current rung and its exit criterion.

Increments

1. A declaration weaker than inferred evidence is accepted silently: effect: read on a tool tagged external_write becomes pass-eligible with zero findings #409 — monotone rule + contradiction finding. Independently shippable; closes the live fail-open in the trust anchor.
2. B — evidence-first effects, questionnaire, progress counter. The activation move.
3. C + F — per-source authority, pack selection at init.
4. E + G — drift pinning, rungs, template mode, CODEOWNERS check.

Headline metric & measurement

Time-to-first-gateable-verdict on an unconfigured real repo (the strategic bottleneck: activation). Secondary: insufficient_evidence rate on the labeled corpus (W27 baseline), and count-of-human-inputs-to-verdict (smart_closer: 48 → target ≤ 7). Re-run both adoption walks (adk-samples#1745, github-mcp-server#3076) as the acceptance harness — they are the measured baseline.

Non-goals & compatibility

• No new CLI commands (surface discipline): everything extends scan, init, apply-patches, verify, doctor, policy packs.
• No weakening of P0: prove actual agent-to-tool bindings #268: untrusted third-party content still cannot self-declare; evidence strength is conditioned on tool_sources[].trust, and heuristics still never drive policy (ux(reporting): distinguish heuristic action projections from evidence-backed assertions #357) — they gain only the power to challenge (A).
• Additive manifest fields (tool_sources[].authority, override, basis); existing manifests unchanged. One report-schema bump (effect evidence + questions) and one contract bump (confirm_declarations) — both follow the standard bump checklists.
• Importing scopes from deployment config (Terraform/OAuth manifests) is deliberately deferred; noted as the natural extension of C.

Grounded in: the 2026-08-18/20 adoption walks (#384–#398), the #403 epic and its verification, the #3076 pincer, and the #409 adversarial-declaration experiment. Related open: #396 (reason-text honesty), #398 (name-candidate poisoning), #357 (heuristic vs evidence distinction — A is its complement, not its reversal), #327 (adoption walk composition).