Skip to content

fix(controller): give each SmoothingController its own EMA state - #297

Merged
jeqcho merged 3 commits into
robocurve:mainfrom
Rishet11:fix/smoothing-per-instance-state
Aug 9, 2026
Merged

jeqcho merged 3 commits into
robocurve:mainfrom
Rishet11:fix/smoothing-per-instance-state

Conversation

@Rishet11

@Rishet11 Rishet11 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #296.

What was wrong

SmoothingController kept its EMA state in the per-trial store under a single
module-level key (controller.py:69), read and written destructively
(controller.py:93-95). Two instances active in the same trial therefore shared
one slot: each layer smoothed against whatever the other last wrote, so both
emitted something that is neither layer's intended output — silently, with no
crash or warning.

The class docstring presents this exact composition as the point of the design
("Demonstrates the middleware composition the single-method interface enables"),
and nothing warns against stacking two.

Stacking alpha=0.9 over alpha=0.5 on the stream [1, 5, 2, 8, 3]:

shared store (actual)  : [1.0, 3.0, 2.5,   5.25,  4.125]
independent (expected) : [1.0, 2.8, 2.53,  4.978, 4.2103]
keys in store: ['_smoothing_prev_action']

One key for two controllers, diverging from the second step on.

The fix

Namespace the key per instance via a module-local serial. The contrast that makes
this the right shape: _INFER_KEY is safe to share because it is only ever
appended to (controller.py:65, controller.py:159), so DefaultController
and EnsemblingController coexisting merely add entries. _SMOOTH_KEY was the
only destructively written controller state, which is exactly why sharing it broke.

I used a counter rather than id(self) so the key is deterministic and readable in
a dumped store.

Tests

test_stacked_smoothing_controllers_keep_separate_state stacks two layers and
compares against the independently computed nested EMA. Against unfixed code:

assert np.allclose([1.0, 3.0, 2.5, 5.25, 4.125],
                   [1.0, 2.8000000000000003, 2.5300000000000002, 4.978000000000001, 4.2103])
E   assert False

Worth noting explicitly: my first attempt at this test passed against the broken
code
, because I drove it with ScriptedPolicy, whose oracle emits near-constant
steps — with a constant stream the corruption is invisible. The test needs a
varying action stream, which is why it uses a small local stub policy. I mention it
because it is the kind of regression test that would otherwise look fine and assert
nothing.

Verification

pytest --cov   1137 passed, 5 skipped   Total coverage: 100.00%
ruff format    127 files already formatted
mypy           Success: no issues found in 74 source files
ruff check     2 errors, both pre-existing in plugins/inspect-robots-agent
               (policy.py:283,288 union order) and untouched here

CHANGELOG conflict, deliberately

The ### Fixed insertion points are all currently claimed by open PRs, so this
entry shares an anchor with my #290 — those two conflict in CHANGELOG.md and
nothing else
. I first placed it where it collided with #251, which is approved
and waiting, and moved it specifically so the collision lands on my own newer PR
instead. Verified clean against #250, #251, #256, #286, #288, #293, #295.

Merge #290 or this in any order and I'll rebase the other immediately.

What I deliberately left out

  • No refusal to nest. Making SmoothingController reject an inner
    SmoothingController would also close the bug and is a smaller behavioural
    surface; I went with isolating state because the docstring advertises
    composition. If you'd rather it refuse, say so and I'll swap it.
  • No change to _INFER_KEY or _BUFFER_KEY. They are shared safely for the
    reason above, and touching them would be scope creep.
  • On reachability, plainly: stacking two smoothers is unusual, and I found this
    by reading for state-isolation bugs, not from a failing run. It is latent. I
    think it earns a fix because it is silent and produces wrong actuation, but it is
    not urgent, and I would understand closing it as won't-fix.

The smoothing state lived under one module-level store key, so two
SmoothingController instances in the same trial overwrote each other and
each smoothed against the other's last write. The result is neither
layer's intended output, with no crash or warning.

Namespace the key per instance. _INFER_KEY stays shared because it is only
appended to; this was the one controller state written destructively.

Fixes robocurve#296
Copilot AI lite review requested due to automatic review settings August 4, 2026 16:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@jeqcho jeqcho left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for a really well-executed fix, and for the unusually honest PR write-up — the note about your first test passing against the broken code (constant-stream corruption being invisible) is exactly the kind of detail that saves the next person a wasted afternoon.

What I checked:

  • The bug is real. _SMOOTH_KEY at src/inspect_robots/controller.py:69 is a single module-level key, read and destructively written in next_action (lines 93–95 pre-fix), so two stacked SmoothingControllers share one EMA slot. I reproduced the arithmetic in the issue (#296) against the code and it holds: from step 2 on, each layer smooths against the other's last write. Given the class docstring explicitly advertises middleware composition, fixing rather than forbidding nesting is the right call.
  • The fix is correct and minimal. Per-instance key minted at construction (f"{_SMOOTH_KEY}:{next(_smooth_serial)}") keeps state trial-local via the store rather than moving it onto self — important, since the store is created fresh per trial (rollout.py:267), so a reused controller instance still gets a clean EMA each trial. Storing prev on the instance would have leaked state across trials; this shape avoids that. I also confirmed the store is never persisted to EvalLog, so the process-local serial comment is accurate, and the counter-over-id(self) choice is sensible for readable dumped stores.
  • The test bites. test_stacked_smoothing_controllers_keep_separate_state in tests/test_controller.py uses a varying action stream (chunk length 1, so DefaultController re-infers every step) and compares against an independently computed nested EMA — the failure output against unfixed code in the PR body confirms it would have caught the regression.
  • Hygiene. All CI checks green including the 100% coverage gate, mypy strict over tests, and the branch is mergeable. The contrast note about _INFER_KEY/_BUFFER_KEY being append-only (and therefore safe to share) is a genuinely useful comment for future controller authors.

One non-blocking observation: a copy.deepcopy/pickle of a SmoothingController would duplicate its store key and reintroduce sharing between the copies. Nothing in the codebase does that today, so I don't think it needs handling — just flagging it in case controller cloning ever becomes a pattern.

Also appreciated the deliberate CHANGELOG anchor choice so the conflict lands on your own #290 rather than #251. LGTM.

jeqcho and others added 2 commits August 8, 2026 17:18
Resolve CHANGELOG conflict: keep both the SmoothingController entry (robocurve#296)
and the duplicate scene id entry (robocurve#289) under Unreleased > Fixed.

Co-Authored-By: Claude Fable 5 <[email protected]>

@jeqcho jeqcho left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after branch update -- CI green on the updated head.

@jeqcho
jeqcho merged commit 2b53533 into robocurve:main Aug 9, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Two SmoothingControllers in one trial share EMA state and corrupt each other

3 participants