Skip to content

Commit 7d365a3

Browse files
authored
Merge pull request pingdotgg#199 from Yarden-zamir/fix/fork-repair-one-thread
fix(fork): repair agent runs in one thread and compacts first
2 parents 3e97b82 + c8bcf9c commit 7d365a3

4 files changed

Lines changed: 26 additions & 17 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ This is a personal fork of [pingdotgg/t3code](https://github.com/pingdotgg/t3cod
4545
- **Changelog:** a timeline of fork releases and commits, also in the update popover for the offered update, the pending upstream commits, a "Start Fork Sync" button and an "Ask agent" menu (`forkChangelogEnabled`, on by default).
4646
- **Fork Sync:** a daily workflow merges upstream main into fork main. It never rewrites main.
4747
- **Fork CI:** runs on each push to main and on each pull request to main. Branch protection requires the one check "Fork CI result".
48-
- **Repair agent:** a failed Fork Sync or Fork Release run, or a failed Fork CI run on main, calls the webhook task "Fork repair" on the VPS, which starts a repair agent. A pull request never calls it.
48+
- **Repair agent:** a failed Fork Sync or Fork Release run, or a failed Fork CI run on main, calls the webhook task "Fork repair" on the VPS. The repair agent works in one long-lived thread and compacts its context after each repair. A pull request never calls it.
4949

5050
**Agents**
5151

‎docs/fork-agents.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,10 +94,10 @@ To restore, ask the owner first. Then:
9494

9595
## Repair agent
9696

97-
When Fork CI, Fork Sync or Fork Release fails, `scripts/fork-notify-agent.sh` posts the run to the webhook in the GitHub secret `T3_AGENT_WEBHOOK_URL`. The webhook is the scheduled task "Fork repair" on the VPS, in the project "t3code fork". It starts a repair agent there.
97+
When Fork CI, Fork Sync or Fork Release fails, `scripts/fork-notify-agent.sh` posts the run to the webhook in the GitHub secret `T3_AGENT_WEBHOOK_URL`. The webhook is the scheduled task "Fork repair" on the VPS, in the project "t3code fork". It posts each run into one long-lived thread, "Fork repair", and the repair agent works there. As its last action, each run queues `/compact` into its own thread with `t3_thread_send` (mode `queue`). The compaction runs as its own turn after the run, so the next repair starts with compacted context. A turn cannot compact itself: the orchestrator rejects `/compact` as a steer.
9898

9999
- The repair agent watches the next run of the same workflow.
100100
- GitHub can start the daily Fork Sync hours after its 03:17 UTC schedule. Before you merge upstream by hand, check `gh run list --workflow fork-sync.yml` and the VPS for a running "Fork repair" thread. If the daily run did not start yet, run the workflow instead of a manual merge. If a repair agent already merges upstream, send it your notes instead of a second merge.
101101
- On a retry, run the workflow with `notify_agent=false`, so that a second failure does not start another agent.
102-
- A failure right after a failure of the same workflow does not post, when the notify job of the first failure succeeded.
102+
- A failure right after a failure of the same workflow does not post, when the notify job of the first failure succeeded and the first failure started less than 6 hours before.
103103
- To stop the repair agent, disable the "Fork repair" task on the VPS (`machines_command` with `{"_tag":"setScheduledTaskEnabled","taskId":...,"enabled":false}`). To stop all calls, delete the GitHub secret `T3_AGENT_WEBHOOK_URL`.

‎docs/fork.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ Fork main only moves forward. Nothing rebases or force-pushes it.
2121
- `CLI_RELEASE_REPOSITORY` in `packages/shared/src/cliRelease.ts` points the server updaters at the fork releases. `scripts/install.sh` installs from the fork releases, on the nightly channel by default.
2222
- A manual run from a branch other than main does not publish a release. It uploads the files of each platform as workflow artifacts, for a test of release changes.
2323
- The release carries `nightly-mac.yml`, `nightly.yml` (Windows), `nightly-linux.yml` (x64) and `nightly-linux-arm64.yml` for electron-updater. Until a platform adds its file, its apps find no update in the new release.
24-
- If a sync, a release or a Fork CI run fails, its `notify_agent` job runs `scripts/fork-notify-agent.sh`. The script posts the run to `T3_AGENT_WEBHOOK_URL`, a webhook scheduled task on a T3 server that starts a repair agent. A manual run with `notify_agent=false` does not post. A failure right after a failure of the same workflow does not post either, when the notify job of the first failure succeeded, because the agent that it started watches the next run. The script retries the call once a minute for 15 minutes, so a webhook server restart does not lose it.
25-
- The webhook is the scheduled task "Fork repair" on the VPS T3 server (`yarden-zamir-vps-2`), in the project "t3code fork" (`/root/Github/t3code/main`). The VPS auto-updates once per release, and each update restarts its server for about 20 seconds, which the 15-minute retry covers.
24+
- If a sync, a release or a Fork CI run fails, its `notify_agent` job runs `scripts/fork-notify-agent.sh`. The script posts the run to `T3_AGENT_WEBHOOK_URL`, a webhook scheduled task on a T3 server that starts a repair agent. A manual run with `notify_agent=false` does not post. A failure right after a failure of the same workflow does not post either, when the notify job of the first failure succeeded and the first failure started less than 6 hours before. The agent that it started watches the next run. The script retries the call once a minute for 15 minutes, so a webhook server restart does not lose it.
25+
- The webhook is the scheduled task "Fork repair" on the VPS T3 server (`yarden-zamir-vps-2`), in the project "t3code fork" (`/root/Github/t3code/main`). The task posts each run into one long-lived thread, "Fork repair". As its last action, each run queues `/compact` into its own thread with `t3_thread_send` (mode `queue`). The compaction runs as its own turn after the run, so the next repair starts with compacted context. A turn cannot compact itself: the orchestrator rejects `/compact` as a steer. The VPS auto-updates once per release, and each update restarts its server for about 20 seconds, which the 15-minute retry covers.
2626
- Rollback: to stop the repair agent, disable the "Fork repair" task on the VPS. To stop all calls, delete the GitHub secret `T3_AGENT_WEBHOOK_URL`.
2727
- `.github/workflows/fork-ci.yml` runs the `ci.yml` commands on GitHub-hosted runners for each push to main. `ci.yml` needs upstream's Blacksmith runners, so it is disabled on the fork.
2828
- Fork CI also runs for each pull request to main. A newer push to main or to a pull request cancels the Fork CI run of the older push. The repair agent ignores a cancelled run.

‎scripts/fork-notify-agent.sh‎

Lines changed: 21 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -11,22 +11,31 @@ if [[ -z "${WEBHOOK_URL:-}" ]]; then
1111
fi
1212

1313
# A repair agent's push to main starts new runs that cannot pass notify_agent=false.
14-
# So a failure right after a failure of the same workflow does not start another
15-
# agent, when the notify job of that failure succeeded: the agent that it started
16-
# watches the next run and reports it. A notify job that failed reached no agent.
17-
# Limit: a skip also ends the job with success, so a chain of failures stays quiet
18-
# after one delivered call. Pull request runs never notify, so they do not count.
14+
# So a failure right after a failure of the same workflow does not post again, when the
15+
# notify job of that failure succeeded and that failure started less than 6 hours before
16+
# this run: the repair agent watches the next run and reports it. A notify job that
17+
# failed reached no agent. Pull request runs never notify, so they do not count.
18+
# Limit: a chain of failures inside the 6 hours stays quiet after one delivered call.
19+
# Revisit the 6 hours if a repair and its check take longer, or if a quiet failure stays unreported.
20+
max_quiet_seconds=$((6 * 60 * 60))
1921
previous="$(gh run list --repo "$GITHUB_REPOSITORY" --workflow "$GITHUB_WORKFLOW" --status completed --limit 20 \
20-
--json databaseId,conclusion,event --jq "[.[] | select(.databaseId != ${GITHUB_RUN_ID} and .event != \"pull_request\")
21-
| select(.conclusion != \"cancelled\" and .conclusion != \"skipped\")][0] // empty | \"\\(.databaseId) \\(.conclusion)\"")"
22-
if [[ "${previous#* }" == "failure" ]]; then
23-
notified="$(gh run view "${previous%% *}" --repo "$GITHUB_REPOSITORY" --json jobs \
22+
--json databaseId,conclusion,event,startedAt --jq "[.[] | select(.databaseId != ${GITHUB_RUN_ID} and .event != \"pull_request\")
23+
| select(.conclusion != \"cancelled\" and .conclusion != \"skipped\")][0] // empty
24+
| \"\\(.databaseId) \\(.conclusion) \\(.startedAt)\"")"
25+
read -r previous_id previous_conclusion previous_started <<<"$previous"
26+
if [[ "${previous_conclusion:-}" == "failure" ]]; then
27+
notified="$(gh run view "$previous_id" --repo "$GITHUB_REPOSITORY" --json jobs \
2428
--jq '[.jobs[] | select(.name == "Notify repair agent")][0].conclusion // empty')"
25-
if [[ "$notified" == "success" ]]; then
26-
echo "The previous ${GITHUB_WORKFLOW} run failed too, and its call succeeded. No new agent is notified."
29+
this_started="$(gh run view "$GITHUB_RUN_ID" --repo "$GITHUB_REPOSITORY" --json startedAt --jq .startedAt)"
30+
# Separate assignments, so that set -e stops the script when a time does not parse.
31+
this_epoch="$(date -d "$this_started" +%s)"
32+
previous_epoch="$(date -d "$previous_started" +%s)"
33+
gap_seconds=$((this_epoch - previous_epoch))
34+
if [[ "$notified" == "success" ]] && (( gap_seconds < max_quiet_seconds )); then
35+
echo "The previous ${GITHUB_WORKFLOW} run failed ${gap_seconds} s before this run, and its call succeeded. No new call."
2736
exit 0
2837
fi
29-
echo "The previous ${GITHUB_WORKFLOW} run failed too, but its notify job ended with '${notified:-none}'. Notifying."
38+
echo "The previous ${GITHUB_WORKFLOW} run failed ${gap_seconds} s before this run, and its notify job ended with '${notified:-none}'. Notifying."
3039
fi
3140

3241
payload="$(jq -n \

0 commit comments

Comments
 (0)