Questionnaire ordering ranks questions that already carry a proposed answer above the blanks that move the verdict #419

Summary

The questionnaire tells the reader it is ordered by how much answering can move the verdict — "money, outward communication, and destruction" first. It ranks by the effect the scan already inferred, so an action with no effect evidence sorts near the floor. Since a pre-filled proposal is also offered only where evidence was observed, the two mechanisms are driven by the same signal: the questions that arrive with a proposed answer systematically outrank the questions that arrive blank.

On the reference walk this puts three already-drafted mail tools at Q2–Q4 and the financial write — the single question that produced both critical blockers — at Q6.

Observed

google/adk-samples#1745, python/agents/smart_closer, CLI 0.16.0b7 at aa6bd618 (post-#416/#417), 13 questions:

  Q  dim        subject                              effect line
  1  authority  adk_agent [tool_source]              —
  2  effect     get_manager_email                    external_communication   ← proposed
  3  effect     list_messages                        external_communication   ← proposed
  4  effect     send_email                           external_communication   ← proposed
  5  effect     create_salesforce_quote              <REVIEW_REQUIRED>
  6  effect     create_sap_sales_order               <REVIEW_REQUIRED>        ← both criticals
  7  effect     get_opportunity_line_items           <REVIEW_REQUIRED>
  …
 13  effect     update_opportunity_status            <REVIEW_REQUIRED>

Answering Q6 financial_write yields:

critical SHIP-POLICY-APPROVAL-MISSING                create_sap_sales_order lacks a declared approval policy
critical SHIP-ACTION-FINANCIAL-WRITE-CONTROL-MISSING create_sap_sales_order has financial write capability without required controls

A reader working top to bottom confirms three drafts before reaching it.

Mechanism

src/agents_shipgate/core/declaration_questions.py:340:

rank = effect_evidence_rank(assessment.conservative_effect)

effect_evidence_rank (core/semantic_assessment.py:1087-1095) is _EFFECT_RANK.get(effect, 0) — documented as sorting unknown values "at the floor rather than raising", correct for a display order that must not fail a scan. And where an action has no effect claims at all, conservative falls back to "write" (semantic_assessment.py:459), rank 1.

So:

The ranking is faithful to observed risk. The problem is that observed risk is not the same quantity as how much an answer can move the verdict, and the header claims the latter. An unobserved action is not low-risk; it is unmeasured, and it is exactly where a human answer carries the most new information.

Why it compounds

A proposal is offered "only where something was observed" (#416). That is the right rule for proposals. But it means proposal-availability and rank share one input, so the ordering concentrates the cheapest questions at the top and defers the most valuable ones — the inverse of the header's promise, and the inverse of what an adopter with limited attention needs.

Suggested direction

Rank blanks above proposals within the same tier, or fold "does this question already carry an answer" into the sort key ahead of inferred rank. A name-shaped hint (create_*, *_order, *_payment, delete_*) as a tiebreaker for zero-evidence actions would go further, and would not need to be trustworthy — it orders questions, it never establishes an effect.

Acceptance

• On the reference walk, the first effect question is one with no proposal.
• A question carrying a proposal never precedes a blank question of equal or lower inferred rank.
• The header sentence and the realised order agree, asserted on a fixture with both kinds present.

Increment 2 follow-up on #410. Found 2026-08-24 on the fifth adoption walk of adk-samples#1745; walk numbers are in #410 (comment).