Skip to content

feat(testing): prepare owned targets and snapshot pinned PR comparisons - #6518

Merged
Rishet11 merged 11 commits into
feat/agentic-pr-testingfrom
okao/ao-agentic-testing-74/setup-intake
Oct 10, 2026
Merged

Rishet11 merged 11 commits into
feat/agentic-pr-testingfrom
okao/ao-agentic-testing-74/setup-intake

Conversation

@Rishet11

@Rishet11 Rishet11 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Code changes: +1021 -95
Tests: +1155 -28
Others: +225 -211

ao testing start --pr <GitHub URL> resolves the configured repository, saves the title, body and full patch, and pins both revisions before creating a comparison pair. Fork heads are fetched by exact SHA into a leased warm checkout. Intake preserves dependencies and registered repository configuration.

For the AO desktop recipe, the controller builds each revision's daemon into a private attempt directory and launches Electron through its checked override. Preflight keeps exact errors. The launcher checks executable, cwd, PID and kernel birth time, guards the live AO daemon, and removes owned preparation state when admission fails. Diagnostics and warm code remain. Changed directory identities block cleanup.

Preparation owns its checkout lease before Python starts. Cancellation stops the private process groups for dependency installation and compilation, verifies their absence, and removes only the matching lease. Failed shutdown observations return a registered partial target and retain the exact lease identity. Final log collection accepts that owned identity before Electron exists. Failed cleanup retains state; a later cleanup completes only after process-group absence is proved. The native desktop factory is deferred until first use, so its platform/path validation cannot block ordinary daemon startup. Factory errors retain their cause. Testing input requires foreground delivery, including the recorded recipe and action policy.

The testing skill reads pinned repository instructions, chooses UI actions or scoped backend/CLI tests from the diff, and keeps base/head in one worker. It waits for cleanup and recording receipts before copying evidence. The worker context uses the agreed JSON tags and checked launch facts. The tester finishes with an AO verdict, comparison artifact and draft review comments for the user; it never posts to GitHub.

Validation: complete scoped tests passed for local target, testing service, CLI, HTTP/API/controllers, daemon, ports, domain and skill assets. Fresh local-target, CLI, daemon and skill suites passed after the follow-ups, as did the full native CLI suite on macOS. Final golangci-lint 2.13.2 reported zero issues. Regressions reproduced the constructor failure and preparation/build cancellation leaks against the published code, then passed after the fixes with checkout reuse. The final ownership regression also failed against the previous head, then passed with the fix. Complete local-target tests and race tests pass on the final source; the full pinned lint check reports zero issues. Guard tests cover inherited AO_PORT=0, a live-port collision and live-daemon birth changes during admission and cleanup. Each follow-up had fresh tests for its touched packages before committing. Tests used private AO data/run-file, CI=true and a short TMPDIR. API spec and frontend types were regenerated together. Three real revision launch probes passed the Phase 0 gate.

Hook exception: the enabled hook reproduced the approved OpenCode, gitdefault and importer host failures documented on clean main. Its additional LAN telemetry EOF was in the unchanged root HTTP package; that test passed 1,000 isolated repetitions and the fresh full package run. This qualifies under the user's timing-failure rule. The hook and test timeouts are unchanged. Each bypass followed fresh tests for that commit's touched packages, recorded in worker A's Phase 1 report. New lint findings were fixed and the full pinned lint check passed.

Known gaps: B's separate PR provides the comparison route, same-worker leg lifecycle and session tools. C's separate PR provides capture reliability. The integrated one-command base/head/cancel live proof is pending. Intake accepts any configured GitHub repository; only the AO target recipe is implemented in this phase. Full local backend race and frontend validation remain pending the serialized validation slot. All 23 CI jobs passed on the prior ed81d4c head, including native macOS and Windows CLI. The second native review identified the failed-preparation ownership gap, fixed in this head with its regression. Final-head native review is approved, and all 23 CI checks passed, including native macOS and Windows CLI. This PR stays draft until those checks and integration are complete.

@github-actions

Copy link
Copy Markdown
Contributor

🏆 Review leaderboard

Oct 3–10, 2026 · UTC

Rank Reviewer PRs reviewed Review rounds PR comments
🥇 @illegalcall @illegalcall 28 47 11
🥈 @ronishrohan @ronishrohan 20 23 17
🥉 @Vaibhaav-Tiwari @Vaibhaav-Tiwari 15 19 11
4 @Prasad-D-Ware @Prasad-D-Ware 10 27 1
5 @nikhilachale @nikhilachale 9 22 3
6 @axisrow @axisrow 8 10 44
7 @mohakchakraborty2004 @mohakchakraborty2004 7 7 1
8 @Rishet11 @Rishet11 6 15 2
9 @Pritom14 @Pritom14 5 5 3
10 @codebanditssss @codebanditssss 4 4 2

Ranked by distinct external PRs reviewed, then review rounds, then PR comments. Self-activity and bot activity are excluded.

Show 6 more reviewers
Rank Reviewer PRs reviewed Review rounds PR comments
11 @Annieeeee11 @Annieeeee11 3 6 1
12 @harshitsinghbhandari @harshitsinghbhandari 2 4 0
13 @neversettle17-101 @neversettle17-101 2 3 0
14 @whoisasx @whoisasx 2 2 0
15 @AgentWrapper @AgentWrapper 1 3 0
16 @ApexYash11 @ApexYash11 1 2 0

@Rishet11 Rishet11 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested for three inline findings: ordinary daemon startup can fail on a long data path, cancelled preparation can strand checkout leases and child processes, and the required lint job fails on new code.

Reviewed the pinned diff at 69be75f07c6aa8ba0d13dc9c35e58a8c8536f872, including intake, API/CLI contracts, target setup, readiness, cleanup and worker guidance. No local tests or builds were run under the AO reviewer rules. Remote lint and native macOS/Windows jobs failed; other checks were still running when inspected.

The comparison and leg endpoints are a declared dependency on the separate lifecycle PR. The complete URL-to-comparison workflow remains unverified.

@Rishet11 Rishet11 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested for one remaining cleanup finding. The lazy desktop factory addresses the earlier daemon-startup issue, and the latest lint and native macOS/Windows checks pass. The cancellation fix covers normal shutdown, but failed preparation shutdown checks lose ownership and can produce a false complete cleanup receipt.

Reviewed ed81d4c9e03d605335db7b47dff3642d1835315f against the PR base, including the four follow-up commits and their regression tests. No local tests or builds were run under the AO reviewer rules. Several other CI checks were still running when inspected.

The comparison/leg endpoints remain a declared dependency on the separate lifecycle PR. The complete URL-to-comparison workflow remains unverified.

@Rishet11 Rishet11 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. No required changes found at 95329faeb0e18f2e2f049c6663959e42eb65e5a1.

The failed-preparation path now returns a registered target identity and retains the matching lease. Cleanup can recheck process absence and retry without reporting completion while ownership is lost. I reviewed the preparation/build recovery tests, log handling and live-daemon guards against the PR base.

I did not run local tests or builds under the AO reviewer rules. Current-head CI has some passing checks; the rest were queued or running when inspected. The declared comparison/leg integration and combined live verification remain pending in the related PRs.

@Rishet11
Rishet11 marked this pull request as ready for review October 10, 2026 13:48
Copilot AI balanced review requested due to automatic review settings October 10, 2026 13:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Rishet11
Rishet11 merged commit 0d76cb0 into feat/agentic-pr-testing Oct 10, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants