Skip to content

A declaration weaker than inferred evidence is accepted silently: effect: read on a tool tagged external_write becomes pass-eligible with zero findings #409

Closed

Description

@pengfei-threemoonslab

Summary

A human declaration that contradicts the scanner's own derived evidence is accepted silently, closes the very gap that evidence raised, and makes the action pass-eligible. Declaring effect: read on a tool shipgate itself tagged external_write produces zero findings — the declaration wins, the evidence vanishes from the decision, and the report still carries the contradicting risk_tags on the same catalog row.

This is the trust anchor failing open. The whole declaration model rests on "a human said so" outranking "the scanner guessed" — which is right — but nothing tells the human, or their reviewer, that what they said is contradicted by what the scanner observed. The path of least resistance for anyone the gate is blocking (including an agent-assisted human rubber-stamping a drafted manifest) is to declare everything read/none, and shipgate will convert that into pass-eligibility without objection.

Reproduction

CLI 0.16.0b7 at 4e712f54 (post-#403/#404), contract 24, report schema 0.35. Workspace: adk-samples#1745 python/agents/smart_closer with the standard scoped init manifest (12 tools, all reachable, declared_purpose filled).

Baseline scan: 24 semantic gaps, 0/12 actions pass-eligible. send_email's gaps include inferred_effect_only ("tool effect is supported only by heuristic evidence", source_ref: risk_hint) and missing_authority_evidence. Its catalog row carries risk_tags: ["customer_communication", "external_write"].

Add a deliberately false declaration:

action_surface:
  actions:
    - tool: send_email
      effect: read
      authority:
        mode: none

Re-run scan. Actual result:

  • Decision text moves on to the next tool (create_salesforce_quote); send_email is done.
  • 1/12 actions pass-eligible (was 0/12); gaps 24 → 22.
  • The pre-existing inferred_effect_only gap on send_email is closed by a declaration that contradicts the heuristic that raised it.
  • Grep of the full report.json for any conflict/contradiction/mismatch mention involving send_email: 0 hits.
  • The same report still contains risk_tags: ["customer_communication", "external_write"] on the send_email catalog row. The contradiction is present in the artifact; nothing joins it.

Mechanism

The contradiction check exists and is correct for the claims it can see:

  • core/semantic_assessment.py:462-495 — when a declaration is present, contradictory = [claim for claim in [*structural, *inferred] if claim.policy_eligible and _EFFECT_RANK[claim.value] > _EFFECT_RANK[declared_effect]]; a hit emits conflicting_effect_evidence ("declared effect is weaker than high-confidence source evidence").
  • core/domain.py:126 — policy_eligible = basis in POLICY_ELIGIBLE_EVIDENCE_BASES and confidence == "high".

send_email's external_communication claim comes from a name-heuristic risk hint (semantic_assessment.py:386-412), so it is not policy_eligible — and is therefore invisible to the contradictory filter. _EFFECT_RANK (:25-35) ranks external_communication at 6 vs read at 0; the rank comparison would have caught it if the claim had been admitted.

The exclusion is a deliberate #357 boundary: heuristic evidence must not drive policy (must never prove a read-only action or independently block). That is right. But the flag conflates two different powers:

  1. Driving a verdict — heuristics rightly cannot.
  2. Challenging a human assertion — heuristics should. A declaration sitting below an inferred claim is not the heuristic gating anything; it is a human statement contradicting an observation, which is precisely the situation a reviewer needs surfaced.

In #403's language: closing inferred_effect_only via a contradicting declaration is a narrowing decision with no record.

Expected behaviour

A declaration ranked below an inferred (non-policy-eligible) claim should:

  • be accepted as the operative effect (human authority stands — heuristics still don't drive policy), but
  • emit a distinct review-level finding (e.g. declaration_below_inferred_evidence, severity review_required, never blocking) naming the declared value, the inferred value, and the hint source, and
  • appear in the PR-comment / reviewer projection as an explicit override row (⚠ "declares read; heuristic evidence says external_communication (risk_hint: name)"),
  • with an explicit override: {evidence, reason} in the declaration silencing nothing but downgrading the finding to an acknowledged override (still visible, never silent).

Acceptance criteria:

  • The reproduction above yields a finding naming send_email, read, and external_communication; the action may remain pass-eligible only if declarations-with-acknowledged-override are pass-eligible by policy.
  • conflicting_effect_evidence behaviour for policy-eligible claims is unchanged (regression: existing tests).
  • A declaration ranked above all evidence (conservative escalation) stays silent — the rule is monotone, only de-escalation is flagged.
  • Negative control: the finding does not fire when declaration and heuristic agree.

Relationship to the RFC

This is Increment 1 of the evidence-first declaration workflow RFC (filed separately): the monotone declaration rule. It is independently shippable and closes the live hole; the RFC generalises the same comparator into drift detection and the proposal/confirmation flow.


Found 2026-08-23 while testing the #403 fixes, via a deliberate adversarial declaration on the smart_closer walk.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Release-blocking: correctness of the gate or the primary adoption patharea:verifyverify / preview orchestration and SHIP-VERIFY-* checksbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions