Skip to content

google_adk.tool_inventories duplicates catalog entries and does not close the incomplete_surface gap it is prescribed for #386

Closed

Description

@pengfei-threemoonslab

Impact

The remediation the tool prescribes for incomplete_surface does not close the gap, and
makes the headline coverage metric worse.
A user who follows the instruction exactly is
left further from a pass than before they started.

This is on the critical path of every ADK adoption: incomplete_surface fires for all six
statically-extracted tools on first scan, and tool_inventories is the only remedy offered.

Severity: P1

Not a cosmetic issue — the loop is unclosable. The instruction is emitted by the tool, the
user complies, and the same verdict comes back with a degraded number. There is no third
step offered.

Reproduction

CLI 0.16.0b7, contract 24. Subject: google/adk-samples#1745,
python/agents/smart_closer.

The verdict instructs:

"Review the skeleton written next to report.json, save it in your repo, reference it from
google_adk.tool_inventories in shipgate.yaml, then rerun verification."

Doing exactly that:

cp agents-shipgate-reports/suggested-inventory.json ./tool-inventory.json
cat >> shipgate.yaml <<'YAML'
google_adk:
  tool_inventories:
    - path: tool-inventory.json
YAML
agents-shipgate scan --format json

Observed

before after
catalog tools 12 18
reachable 6 6
ratio 6/12 6/18
incomplete_surface: get_manager_email present still present

The catalog holds 18 entries but only 12 unique names. Every one of the six inventory
tools is double-counted:

list_messages: 2   get_user_identity: 2   get_manager_email: 2
send_email: 2      map_salesforce_account_to_sap_bp: 2
map_salesforce_product_to_sap_material: 2

Removing the tool_inventories block returns the catalog to 12, confirming the inventory is
the cause.

Root cause

google_adk.tool_inventories is list[ArtifactPathConfig], and ArtifactPathConfig has
only two fields:

path: str      (required)
optional: bool (optional)

There is no field binding an inventory to a tool_source. Confirmed by probing
source_id, id, source, tool_source, tool_source_id, for_source, and source_ref
— all rejected with Extra inputs are not permitted.

So an inventory cannot complete the tools of the source that raised the gap; its entries
are appended to the catalog as new tools that happen to share names. Since the gap is keyed
to the original source's tool (get_manager_email [adk_agent]), it is never satisfied.

Expected behavior

An inventory referenced to resolve incomplete_surface for source adk_agent should raise
that source's tools to high confidence, closing the gap, with the catalog remaining at 12.

Acceptance criteria

  • After the documented remediation, incomplete_surface for get_manager_email [adk_agent]
    is closed.
  • Catalog size does not grow when an inventory names tools already extracted from a source.
  • Either a source-binding field exists on inventory entries, or entries are merged by
    (source_id, tool_name).
  • Coverage ratio is monotonic: following a prescribed remediation never lowers it.
  • Regression test asserting the ratio does not decrease across a prescribed fix.

Notes

The monotonicity property in criterion 4 is worth a general guard. A remediation that
degrades the metric it was issued to improve will erode trust in every other number the
report prints.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions