Repository navigation
Description
Impact
The remediation the tool prescribes for incomplete_surface does not close the gap, and
makes the headline coverage metric worse. A user who follows the instruction exactly is
left further from a pass than before they started.
This is on the critical path of every ADK adoption: incomplete_surface fires for all six
statically-extracted tools on first scan, and tool_inventories is the only remedy offered.
Severity: P1
Not a cosmetic issue — the loop is unclosable. The instruction is emitted by the tool, the
user complies, and the same verdict comes back with a degraded number. There is no third
step offered.
Reproduction
CLI 0.16.0b7, contract 24. Subject: google/adk-samples#1745,
python/agents/smart_closer.
The verdict instructs:
"Review the skeleton written next to report.json, save it in your repo, reference it from
google_adk.tool_inventoriesin shipgate.yaml, then rerun verification."
Doing exactly that:
cp agents-shipgate-reports/suggested-inventory.json ./tool-inventory.json
cat >> shipgate.yaml <<'YAML'
google_adk:
tool_inventories:
- path: tool-inventory.json
YAML
agents-shipgate scan --format jsonObserved
| before | after | |
|---|---|---|
| catalog tools | 12 | 18 |
| reachable | 6 | 6 |
| ratio | 6/12 | 6/18 |
incomplete_surface: get_manager_email |
present | still present |
The catalog holds 18 entries but only 12 unique names. Every one of the six inventory
tools is double-counted:
list_messages: 2 get_user_identity: 2 get_manager_email: 2
send_email: 2 map_salesforce_account_to_sap_bp: 2
map_salesforce_product_to_sap_material: 2
Removing the tool_inventories block returns the catalog to 12, confirming the inventory is
the cause.
Root cause
google_adk.tool_inventories is list[ArtifactPathConfig], and ArtifactPathConfig has
only two fields:
path: str (required)
optional: bool (optional)
There is no field binding an inventory to a tool_source. Confirmed by probing
source_id, id, source, tool_source, tool_source_id, for_source, and source_ref
— all rejected with Extra inputs are not permitted.
So an inventory cannot complete the tools of the source that raised the gap; its entries
are appended to the catalog as new tools that happen to share names. Since the gap is keyed
to the original source's tool (get_manager_email [adk_agent]), it is never satisfied.
Expected behavior
An inventory referenced to resolve incomplete_surface for source adk_agent should raise
that source's tools to high confidence, closing the gap, with the catalog remaining at 12.
Acceptance criteria
- After the documented remediation,
incomplete_surfaceforget_manager_email [adk_agent]
is closed. - Catalog size does not grow when an inventory names tools already extracted from a source.
- Either a source-binding field exists on inventory entries, or entries are merged by
(source_id, tool_name). - Coverage ratio is monotonic: following a prescribed remediation never lowers it.
- Regression test asserting the ratio does not decrease across a prescribed fix.
Notes
The monotonicity property in criterion 4 is worth a general guard. A remediation that
degrades the metric it was issued to improve will erode trust in every other number the
report prints.
Activity
fix(inventory): keep source identity and evidence through completion (#…
fix(inventory): alias identity everywhere, and check preserved eviden…
fix(inventory): bind a tool inventory to the source it completes (#386)…
fix(google-adk): prove framework provenance, not just spelling (#400 …
fix(google-adk): prove framework provenance, not just spelling (#400 …
fix(google-adk): measure extraction confidence instead of assuming it (…
fix(inputs): address PR review — decode per language, and bind the ex…
Read an MCP server's tool surface out of its own source (#431) (#483)