Repository navigation
Description
Problem
Every experiment default flip silently misses existing installs because the persisted experiments map records all keys, not just user overrides:
SettingsViewputs the wholeexperimentsmap into theupdateSettingspayload on every Save (webview-ui/src/components/settings/SettingsView.tsx:455).cachedState.experimentsis initialised fromExtensionState, which is the persisted map merged over the defaults.webviewMessageHandlermerges that map overexperimentDefaultand persists the result (src/core/webview/webviewMessageHandler.ts:843-851).experiments.isEnabledtreats a stored value as explicit:experimentsConfig[id] ?? experimentDefault[id](src/shared/experiments.ts:42).
Consequence: a user who saved the settings form once has every experiment key stored at its then-current default. A later default flip in experimentConfigsMap only reaches installs whose persisted map has no value for that key. This is the rollout limitation documented in #1384: making chat-diff the default approval path only reaches installs that have never saved the settings form, and a stored preventFocusDisruption: false is indistinguishable from an explicit opt-out.
Why it matters
Any future experiment graduated by flipping its default gets the same behavior. It also makes rollouts hard to reason about: two users with identical explicit choices can end up with different experiment states depending on whether they ever pressed Save.
Proposed direction
Persist only explicit overrides instead of the merged map. Options:
SettingsViewsends only experiment ids whose value differs fromexperimentDefault(and the handler replaces the stored map instead of merging), or- keep an explicit-overrides set (
experimentsOverrides) and haveisEnabledread that, falling back to the default when absent.
A migration that drops keys equal to the old default is not recommended: it cannot tell a default value from an explicit choice, and would silently re-enable experiments users intentionally disabled.
Acceptance criteria
- A default flip in
experimentConfigsMapreaches installs that have saved the settings form unless the user explicitly toggled that experiment. - Unit tests cover the
SettingsViewpayload (only changed experiment keys), the handler persistence (stored overrides, not merged defaults) andisEnabledprecedence. - No migration that guesses between default and explicit values.
Out of scope for #1384: that PR keeps the current persistence behavior and documents the rollout limitation in its description. Filed from the @edelauna review thread on #1384 (src/shared/experiments.ts:25).
Activity