No sample exercises the declaration questionnaire, so its ordering and rendering are unpinned by any committed artifact #425

Summary

The declaration questionnaire is the primary cold-start adoption surface — it is what an adopter at insufficient_evidence reads, in order, to reach a verdict. No shipped sample exercises it. Every sample in samples/ answers every declaration question it asks, so the ordering, the counter, the DeclarationQuestionRow field shape, and the entire suggested-declarations.yaml renderer are covered only by unit tests and never by a committed artifact.

This is not theoretical. #419 shipped an ordering that put a structurally proven OpenAPI GET named delete_account at the top of the questionnaire, ahead of a genuinely unknown effect, with a repository-chosen name breaking the tie. It was found in review of #421, against a fully green suite: all sample goldens, the markdown byte-comparisons, the Conductor field-for-field check, and the in-tree verifier's passed / mergeable were all structurally incapable of seeing it.

Observed

All 15 samples carrying a manifest, scanned at 40552346:

sample                        decision                questions   open
clean_read_only_agent         passed                  0/0         0
openai_agents_sdk_agent       passed                  0/0         0
agent_weakens_gate            passed                  0/0         0
hitl_evidence_covered_agent   passed                  0/0         0
hitl_evidence_agent           review_required         0/0         0
google_adk_agent              review_required         2/2         0
large_multi_framework_agent   review_required         3/3         0
simple_langchain_agent        passed                  0/0         0
simple_crewai_agent           review_required         0/0         0
simple_openai_api_agent       blocked                 4/4         0
simple_anthropic_agent        blocked                 4/4         0
support_refund_agent          blocked                 2/2         0
conductor_agent               insufficient_evidence   2/2         0
ai_generated_refund_pr        insufficient_evidence   1/1         0

open = 0 everywhere — including both samples that do reach insufficient_evidence, which get there for other reasons. shipgate-self.yaml, which the verify-self CI job runs, reports passed with 0/0.

What that leaves uncovered

• suggested-declarations.yaml has no golden at all. No sample emits the file, so nothing byte-compares the header, the Question N of M banners, the reading lines, the proposal annotations, the unfillable-question notes, or the "Also required, and not a per-action question" section. report.md and packet.* are byte-compared; the file an adopter is actually told to edit is not.
• test_sample_expected_report_json_has_no_structural_drift cannot see DeclarationQuestionRow. It compares field paths, and open_questions is [] in all five goldens, so no open_questions[].* member appears in any of them. Adding, removing, or renaming a field on that model is invisible to the one test built to catch exactly that kind of drift.
• Ordering is unpinned end to end. open_questions[] order, and the _in_question_order permutation of evidence_gaps[] that follows from it — which selects primary_evidence_gap, and so the Improve evidence: line, the decision reason, and first_recommended_action — are exercised only by synthetic in-process fixtures.
• The progress sentence is only ever rendered at zero. Declaration questions: N of M answered; K open (…) reaches the CLI, the file header, and the PR comment from one rendering, and no committed artifact carries it with a non-zero K.

Why it compounds

The walk this whole surface was built for (google/adk-samples#1745) is an external repository. Reproducing it needs a clone that is not in this repo and not in CI, so the only end-to-end evidence that the cold-start path works is a local run somebody remembers to do. Every increment of #410 has been validated that way, and #419 is the first time that gap produced a shipped defect rather than a caught one.

Suggested direction

One sample that stops at insufficient_evidence with open declaration questions, carrying expected/report.json, expected/report.md, and — the point of the exercise — expected/suggested-declarations.yaml as a byte-compared golden.

It should carry all three tiers the ordering distinguishes, because a fixture with only one proves nothing about the order:

• an action nothing was read about (no evidence at all);
• an action with only a protocol default standing in for its absence;
• an action with a heuristic reading the scan may not act on;
• an action the scan read as risky, so its block arrives with a proposed answer;
• a source-wide authority question, so the merged-block numbering is rendered too.

Then extend test_sample_expected_report_json_has_no_structural_drift to it, so open_questions[].* enters the compared path set.

The ADK shape is the one to mirror — samples/google_adk_agent already exists and answers everything, so a sibling that deliberately does not is the smallest honest addition. Whether to grow that sample or add a new one is an open call for whoever picks this up.

Not a new public surface under CONTRIBUTING.md § Surface discipline — no CLI command, schema version, report block, discovery surface, or adapter. It is test evidence for a surface that already shipped.

Acceptance

• At least one sample reports a non-zero declaration_questions.open, and its expected/report.json carries populated open_questions[] rows.
• That sample's expected/suggested-declarations.yaml is committed and byte-compared, the way report.md is.
• test_sample_expected_report_json_has_no_structural_drift covers it, so open_questions[].* is in the compared path set.
• Reordering the questionnaire — for example by reverting _reach in core/declaration_questions.py to rank by conservative_effect — fails at least one committed-artifact test, not only a unit test.

Found while addressing review feedback on #421 (#419).