Skip to content

SetStatusAgentHook never resets _cost on retry attempts, double-counting the previous attempt's spend in the batch status line #1567

Open

Description

@BlueX888

Describe the bug

SetStatusAgentHook (which renders the per-instance status line in the batch progress UI) never resets self._cost when a new attempt starts. On a retry attempt, the status line therefore shows the previous attempt's spend twice — roughly 2x the actual spend — until the new attempt's first step completes.

Root cause in sweagent/agent/hooks/status.py:16-20:

def on_setup_attempt(self):
    self._i_attempt += 1
    self._i_step = 0
    # Costs will be reset for the next attempt
    self._previous_cost += self._cost

The comment says "Costs will be reset for the next attempt", but self._cost is never reset. on_step_start then displays self._previous_cost + self._cost (status.py:28), double-counting the previous attempt's spend until on_step_done (status.py:31) overwrites _cost with the new attempt's instance_cost.

This is reachable in normal batch runs: sweagent/run/run_batch.py:347 registers SetStatusAgentHook for every agent, and with a type: retry agent, RetryAgent._next_attempt (sweagent/agent/agents.py:321) re-attaches the same hook objects to a fresh DefaultAgent (agents.py:312-318), whose setup calls self._chook.on_setup_attempt() (agents.py:594) at the start of every retry attempt.

Expected behaviour: after attempt 1 ends having spent $1.00 in total, the first status line of attempt 2 should read Attempt 2 Step 1 ($1.00). The basis is the code itself: the comment at status.py:19 ("Costs will be reset for the next attempt") and the display formula (${self._previous_cost + self._cost:.2f}) at status.py:28 — the previous+current split only yields the true total spend if _cost is zeroed at the attempt boundary.

Steps/commands/code to Reproduce

Setup from source (no docker needed; plain pip install -e . in a venv), at commit 3ea751c087f32b16e039a2233dd6eefecef325d5 (current main HEAD):

git clone https://github.com/SWE-agent/SWE-agent && cd SWE-agent
python3.12 -m venv .venv && . .venv/bin/activate
pip install -e .
python /tmp/repro_status_hook_cost.py

/tmp/repro_status_hook_cost.py:

"""Repro: SetStatusAgentHook double-counts the previous attempt's cost in the
status line during a retry attempt (comment says "Costs will be reset for the
next attempt", but `self._cost` is never reset in on_setup_attempt).
"""
from sweagent.agent.hooks.status import SetStatusAgentHook
from sweagent.types import AgentInfo, StepOutput

statuses = []
hook = SetStatusAgentHook("inst-1", lambda _id, msg: statuses.append(msg))

# --- Attempt 1: two steps, each costing 0.50 -> total 1.00 ---
hook.on_setup_attempt()
for i in range(2):
    hook.on_step_start()
    hook.on_step_done(step=StepOutput(), info=AgentInfo(model_stats={"instance_cost": 0.5 * (i + 1)}))

print("attempt 1 statuses:")
for s in statuses:
    print("   ", s)

# --- Attempt 2 starts (retry) ---
hook.on_setup_attempt()
statuses.clear()
hook.on_step_start()  # first step of attempt 2, no new cost yet

print("attempt 2, step 1 status:", statuses[-1])
# Actual money spent so far is still $1.00 (attempt 2 has not completed a step).
# Expected: "Attempt 2 Step 1 ($1.00)" -- instead _previous_cost($1.00) + stale _cost($1.00)
assert "($1.00)" in statuses[-1], f"double counted: {statuses[-1]}"
print("PASS: no double counting")

Error message/results

Verbatim output of python /tmp/repro_status_hook_cost.py — attempt 2's first status line shows ($2.00) although only $1.00 has been spent so far:

👋 INFO     This is SWE-agent version 1.1.0
            (hash='3ea751c087f32b16e039a2233dd6eefecef325d5') with SWE-ReX
            version 1.4.0 (rex_hash='unavailable').
attempt 1 statuses:
    Step   1 ($0.00)
    Step   2 ($0.50)
attempt 2, step 1 status: Attempt 2 Step   1 ($2.00)
Traceback (most recent call last):
  File "/tmp/repro_status_hook_cost.py", line 29, in <module>
    assert "($1.00)" in statuses[-1], f"double counted: {statuses[-1]}"
           ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: double counted: Attempt 2 Step   1 ($2.00)

Expected: attempt 2, step 1 status: Attempt 2 Step 1 ($1.00) and PASS: no double counting (resetting self._cost = 0.0 in on_setup_attempt after the accumulation produces exactly that).

System Information

  • SWE-agent 1.1.0, commit 3ea751c087f32b16e039a2233dd6eefecef325d5 (current main HEAD), installed from source via pip install -e . (no docker)
  • Python 3.12.14
  • macOS 15, arm64 (Apple Silicon)

Checklist

  • I'm running with the latest docker container/on the latest development version (i.e., I ran git pull)
  • I have copied the full command/code that I ran (as text, not as screenshot!)
  • If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
  • I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.

Related: I searched the tracker and found no existing report of the status line double-counting spend across retry attempts. #1492 is related in theme (cost accounting around retries) but is a different mechanism — actual re-billing of API calls by the tenacity retry in sweagent/agent/models.py — not the status-line display in sweagent/agent/hooks/status.py.

Happy to open a PR with the approach above (reset self._cost = 0.0 in on_setup_attempt after accumulating it into _previous_cost) — I have verified locally that it makes the repro pass.

Activity

  1. BlueX888 commented on Sep 25, 2026

    @BlueX888
    Author

    Reproduced this. Root cause: sweagent/agent/hooks/status.py:16-21: on_setup_attempt folds the finished attempt's cost into _previous_cost but never resets _cost, so the status line's ${_previous_cost + _cost} (status.py:29) double-counts the previous attempt's spend until the new attempt's first step completes.

    Minimal reproduction:

    python /tmp/repro_status_hook_cost.py
    

    Observed output:

    attempt 1 statuses:
        Step   1 ($0.00)
        Step   2 ($0.50)
    attempt 2, step 1 status: Attempt 2 Step   1 ($2.00)
    AssertionError: double counted: Attempt 2 Step   1 ($2.00)
    

    Happy to open a PR with this fix, or to be assigned if you prefer to take it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions