Summary

azd ai toolbox skill add <toolbox> <skill> creates a new immutable toolbox version, but it branches that new version from the toolbox's default version, not the latest. Sequential add calls therefore produce sibling versions — each containing only one new skill — instead of a chain that accumulates skills. There is no warning, no error, and the CLI output reads as if the operation appended to current state.

The same shape applies to azd ai toolbox skill remove, azd ai toolbox connection add, and azd ai toolbox connection remove (all three call GetToolbox → GetToolboxVersion(tb.DefaultVersion) then construct the new version off that snapshot, e.g. https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_add.go#L132-L134).

Discovered while running an end-to-end test pass against the public toolbox + skills how-to docs on 2026-06-15 (extension azure.ai.toolboxes 0.1.0-preview).

Repro (live, observed 2026-06-15)

Starting state: web-search-toolbox v1 = default, contains web_search tool, no skills. Two skills already exist in the project: greeting-test-2606, code-review-test-2606.

# Step 1 — add greeting
azd ai toolbox skill add web-search-toolbox greeting-test-2606
# Output: Created toolbox web-search-toolbox version 2 (attached skill greeting-test-2606).

# Step 2 — add code-review (intent: now have BOTH skills attached)
azd ai toolbox skill add web-search-toolbox 'code-review-test-2606@1'
# Output: Created toolbox web-search-toolbox version 3 (attached skill code-review-test-2606@1).

# Step 3 — verify
azd ai toolbox show web-search-toolbox --version 3

Actual output (Step 3)

FIELD            VALUE
Name             web-search-toolbox
Default version  1
Shown version    3
Tools            1
Skills           1

SKILL                  VERSION  TYPE
code-review-test-2606  1        skill_reference

greeting-test-2606 is not attached to v3. v3 was branched from v1 (default), not v2 (latest).

Workaround that does what users expect

azd ai toolbox skill add web-search-toolbox greeting-test-2606    # v2
azd ai toolbox publish web-search-toolbox 2                       # promote v2 -> default
azd ai toolbox skill add web-search-toolbox 'code-review-test-2606@1'  # v3 = v2 + code-review (correct)

…or a single --from-file listing both skills.

Why this is a footgun

Every other "add to a list" CLI in the ecosystem (az, gh, kubectl, git) appends to current state. The current behavior is silently lossy:

• No error, no warning at the moment of the wrong outcome.
• The CLI success message (Created toolbox … version 3 (attached skill code-review)) reads as a strict additive operation.
• The user only learns otherwise via toolbox show, after which they have orphan v2/v3/v4 versions to clean up.

The skill-group --help text says "Each add/remove creates a new immutable version; the toolbox's default version is unchanged" — accurate, but most readers parse it as "appends to current state".

Proposed fix

One of:

1. (Preferred) Branch from latest version by default. Add an explicit --from-version <n> flag for users who want to fork from a specific version (current "default-snapshot" semantics still reachable via --from-version <default-version-number>).
2. Reject the second add with an error if default != latest, prompting the user to either publish or pass an explicit version selector.

Apply symmetrically to skill remove, connection add, connection remove.

Stopgap UX changes to ship before / alongside the behavior fix

1. CLI output — surface the trap at the moment of action. Today's source: https://github.com/Azure/azure-dev/blob/main/cli/azd/extensions/azure.ai.toolboxes/internal/cmd/toolbox_skill_add.go#L238-L263. Suggested wording:

   Created toolbox web-search-toolbox version 3 (attached skill code-review-test-2606@1;
     branched from version 1 [default]; latest version is 2 — its skills are NOT carried over).
   Run `azd ai toolbox publish web-search-toolbox 2` first if you wanted to extend the latest version.
   

2. --help text — replace "Each add/remove creates a new immutable version; the toolbox's default version is unchanged" (in toolbox_skill_group.go and the corresponding connection help) with "Each add/remove creates a new immutable version branched from the toolbox's default version. To branch from a different version, use --from-version. The toolbox's default version is unchanged."

3. Public docs — add a Note callout in the toolbox skills how-to "Attach skills" section explaining the semantic and recommending --from-file (single call) or publish-between-adds for accumulation scenarios.

Environment

• azd 1.23.9 (current is 1.25.6 — same behavior expected from source; not yet re-verified end-to-end)
• Extension: azure.ai.toolboxes 0.1.0-preview
• Foundry project: https://zhuoqunli-westus2.services.ai.azure.com/api/projects/proj
• Discovered during: foundry-samples toolbox public-docs test pass on 2026-06-15. Internal cross-reference: finding F-3.1 in internal/findings/TOOLBOX_DOCS_FINDINGS_2026-06-15.md.

Severity / priority

Not a crash, but a silent data-loss-shaped UX trap on a preview feature with growing usage. Recommend addressing before toolbox + skills GA.