feat(ci): sync open issues to a contributor Project with suggested areas and owners by Rishet11 · Pull Request #6995 · OrchestratorInc/agent-orchestrator

Rishet11 · 2026-10-21T00:59:07Z

Code changes: +1902 -0
Tests: +2004 -0
Others: +1477 -0

What

Adds a workflow that keeps a private org GitHub Project of every open issue. Each issue gets a suggested product area and up to three suggested owners. Core contributors filter the board by area, or by their own name, to find work.

Why

About 500 issues are open and nothing says who should look at which one. This gives every issue a starting owner, based on who actually works in that part of the code.

How

The sync runs every 15 minutes and manually. Issue events are reconciled on the next scheduled run.

It adds open issues that are missing from the Project and archives closed ones. If an issue the bot archived gets reopened, it comes back.
It picks an area. First match wins:
- a hand-reviewed route in data/issue-areas.json (about 490 issues)
- a single area:<id> label on the issue
- a text match against the hand-reviewed issues, used only when confident (82% correct in leave-one-out testing, covers 11% of issues)
- otherwise Needs area review
It writes the area's top 3 core contributors into Suggested owner.

Owners rebuild once a day in a separate job with its own concurrency group, and the result is cached by source/data hash and UTC date. Per area, each person scores:

Signal	Weight
Merged PRs touching the area, last 90 days	60%
Lines changed there, older work counting less	25%
Reviews on those PRs	10%
Current lines they wrote, from a blame snapshot (`data/blame-lines.json`)	5%

Core means 10+ merged PRs in 90 days. data/core-overrides.json adds or drops people.

Human edits win. A Sync record field stores what the bot last wrote, and the bot only changes a field that is blank or still holds its own value.

It never assigns, adds labels, comments, or touches items from other repos. The App token is limited to this repo, org Projects write and Issues read.

Review focus: apply_plan in sync_project.py (the write path) and the cache steps in the workflow.

Left out on purpose:

No area:* labels exist yet. A labelling rule for the repo bot is drafted but not enabled, so most new issues land in Needs area review until it is.
Blame is a snapshot from 2026-10-21, refreshed by hand (scripts/blame_snapshot.py, see README). Live blame took over 15 minutes per run.
Removed 10 path rules for agent-switch code that is gone from main.

Scaling and recovery

Each run allows at most 100 mutation attempts, paced at least one second apart, and reserves enough budget to finish an item before starting another. Scheduled runs allow at most 400 attempts per hour. Retries count against the budget.
Pending field targets are recorded before writes. A later run finalizes confirmed values and resumes unfinished updates while preserving human edits. Deferred syncs emit complete: false, a reason and remaining-item count, and exit 3. The workflow shows the JSON summary and a warning. A deferred run that wrote nothing exits 4 and fails the job, so a stalled backlog is visible; other errors also fail the job. A rate-limit wait that would run past 30 minutes defers instead of sleeping into the 45-minute job timeout.
Normal scans read active items and archived open items, leaving closed archive history out of the scan. A separate total count warns at 40,000 items. At GitHub's 50,000-item limit, new issues are skipped, existing items still update, and the job fails with an error until the board is rotated. Rotation to a new private Project is an explicit administrator step; old history is retained and never automatically deleted.
Owner extraction reuses historical PR evidence, refreshes a seven-day overlap, and expires evidence older than 540 days. Full refreshes run every 30 days or on request. Interrupted attempts retain a pinned capture time and completed API pages; inconsistent pagination is discarded and fetched again. The last complete evidence file and successful owners survive failed attempts.
Owner refreshes and sync use independent jobs and concurrency groups. Owner caches hash the owner scripts and data files, excluding sync_project.py, so sync-only edits keep today's owners and generated bytecode cannot change the key between restore and save. PR evidence is saved only when a failed extraction made progress, owners built, or nothing was cached yet, so a failing build does not save a new cache every 15 minutes.

Setup after merge (admin)

Nothing runs until step 1 is done.

Create a private org Project and set the repo variable AO_CONTRIBUTOR_PROJECT_URL.
Create a GitHub App with org Projects write and repo Issues read, installed on this repo. Set AO_CONTRIBUTOR_APP_CLIENT_ID and the secret AO_CONTRIBUTOR_APP_PRIVATE_KEY.
Run the workflow by hand and check its progress summary. The initial backfill continues over scheduled runs; owner suggestions arrive after the separate owner build succeeds.

If the owner build fails or the owners are more than 7 days old, the run shows a warning annotation.

Testing

All 137 Python tests pass, including fault injection before/after each field-write phase, preservation of human edits, cached-page resume, pagination invalidation, and stable cache keys.
A stateful 512-issue backfill completes over 26 runs, each limited to 100 writes. Every completed item has its suggestions and finalized record.
All 19 live GitHub scenarios pass on the private QA Project with disposable issues. This includes a real 10-write stop/resume and recovery from staged partial updates. The Project is closed and the fixture repository archived after testing.
Cold extraction took 370.62 seconds; the first incremental refresh took 29.89 seconds. Both owner builds succeeded.
On 8d1e69ba7f, all 137 Python tests, area-map validation and workflow lint pass. Six further live checks pass: dry-run, the 10-write limit, resume, existing-item updates while a lowered test capacity blocks additions, a later addition after that threshold clears, and a zero-write rerun. The real API was used for reads and writes; the capacity threshold alone was lowered in process. The private QA Project is closed and the fixture repository archived again.
All GitHub checks pass on this head: frontend tests, Linux browser integration, renderer smoke and Mac helpers, contributor tests, and Gitleaks. The privileged owner/sync jobs correctly skip PR runs.
Native validation also passed the full frontend suite on retry, package checks, docs build and helper builds. The local workflow emulator failed on worktree/Git transport and its Playwright image loader. The native macOS browser integration has the same drag-region failure on this head and unchanged fc43317; the corresponding Linux integration passed in GitHub CI.

The narrowed App installation token and hosted Actions cache restore/save path have not been exercised live. The live sync checks used the authorized CLI OAuth token. Production Project settings and existing issues were untouched.

Checklist

…eas and owners Adds a GitHub Actions workflow and Python helpers that keep a private GitHub Project of open issues for core contributors. Each issue gets a Suggested area (28 areas, mapped from file paths) and a Suggested owner (ranked from merged-PR history and git blame per area). The sync only adds open issues and fills blank suggestion fields. It never sets assignees, labels or human-edited values, defaults to a dry run, and is skipped until AO_CONTRIBUTOR_PROJECT_URL is configured.

github-actions · 2026-10-21T00:59:38Z

🏆 Review leaderboard

Oct 14–21, 2026 · UTC

Rank	Reviewer	PRs reviewed	Review rounds	PR comments
🥇	@axisrow	295	415	5
🥈	@nikhilachale	25	45	8
🥉	@ronishrohan	23	25	15
4	@Prasad-D-Ware	20	53	6
5	@illegalcall	19	76	13
6	@Vaibhaav-Tiwari	14	16	7
7	@whoisasx	14	15	1
8	@Rishet11	9	40	21
9	@somewherelostt	6	18	0
10	@Annieeeee11	6	10	2

Ranked by distinct external PRs reviewed, then review rounds, then PR comments. Self-activity and bot activity are excluded.

Show 7 more reviewers

Rank	Reviewer	PRs reviewed	Review rounds	PR comments
11	@mohakchakraborty2004	6	6	0
12	@Pritom14	3	3	5
13	@codebanditssss	2	3	0
14	@neversettle17-101	2	3	0
15	@wesleymatosdev	1	2	0
16	@harshitsinghbhandari	1	1	0
17	@LaibaFirdouse	1	1	0

The --repo default was computed from ROOT on every run, so tests that patch ROOT to a shallow temp dir (e.g. /tmp on Linux runners) raised IndexError. Resolve it once from the script's location, matching ownership_extract.py.

…oject sync Adding an issue that is already on the Project returns the existing item, so when the Project item list lags, an issue that looked new could already hold a human value. Recheck every item before filling, not only pre-existing ones. One timeout, dropped connection, HTTP 5xx or GraphQL internal error aborted a run of ~1,500 calls. Retry those once after 5 seconds, with a budget separate from the rate-limit retry, and log each retry.

- Route issues without a hand-reviewed area by a single area:<id> label, then a confident text match, else Needs area review. - Rebuild owners daily in CI from merged PRs and reviews plus a committed blame snapshot, instead of committing the owner map. - Record bot-written values in a Sync record field so they refresh while human edits stay untouched. - Archive closed issues and bring back reopened ones the bot archived. - Sort and dedupe issue pages, limit the App token to what the sync needs, warn when the owner build fails or goes stale, and compact the data maps.

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

ronishrohan

Live verification found two issues to fix before merge: the P1 reopen failure and the P2 area/owner mismatch below.

Nine of eleven live scenarios passed. All 114 existing Python tests and a fresh owner-data build passed. Live testing used a private scratch Project and disposable issues in a private test repository. Production Project settings, existing issues and PR code were untouched; the QA Project is now closed and the test repository archived.

The narrower App installation token and the GitHub Actions cache lifecycle were not exercised. Live sync used an authorized CLI OAuth token.

… area The unarchive mutation was built by chained replaces on the archive one, which produced UnunarchiveProjectV2ItemInput and stopped every later sync. Write it out and test both mutation names. When a human sets Suggested area, suggest owners for that area. Unowned areas clear only a bot-written owner; unknown options leave it alone.

ronishrohan

Approved after retesting fc43317.

All 116 Python tests, 10 local CLI scenarios, and 14 live GitHub scenarios passed. Area-map validation passed too.

Manually verified dry-run safety, initial population, unchanged reruns, human area and owner edits, owned/unowned/custom area routing, manual archives, close/reopen recovery, draft preservation, and invalid-owner fallback. Both reported bugs are fixed: reopened issues return correctly and the next sync makes zero writes; owner suggestions follow the preserved human-selected area.

Live checks used a private QA Project and disposable test issues. The Project is closed and the fixture repository archived after testing. Production settings and existing issues were untouched.

No remaining blockers from this retest. The full repository CI job is still running, so wait for that to pass before merging. The narrower App installation token and the Actions cache lifecycle were not exercised live.

Withdrawing approval while I address bulk-sync rate limits, interrupted-write recovery, archive growth, incremental owner extraction, and owner-build scheduling.

- Exit 4 and fail the job when a deferred sync wrote nothing, so a stuck backlog is visible instead of staying green. - At capacity, skip only new adds; existing items still update and the job fails with an error until the board is rotated. - Defer rate-limit waits that would run past 30 minutes, before the 45-minute job timeout. - Save PR evidence only when a failed extraction made progress, owners built, or nothing was cached yet. - Key the owner cache on owner scripts and data, not sync_project.py.

Rishet11 · 2026-10-24T21:56:03Z

@ronishrohan thanks for the scaling work. I pushed 8d1e69ba7f on top of 73dd92e196 to cover five smaller gaps in it:

Stalled backlog was green. A deferred run that wrote nothing now exits 4 and fails the job. Exit 3, which means partial progress, still passes with a warning.
Capacity aborted the whole run. At 50,000 items, sync now skips only new adds. Existing items still get unarchived and filled, then the job exits 2 with an error until the board is rotated.
Rate-limit waits were unbounded in total. A wait that would run past 30 minutes into the sync now defers, so the job exits cleanly before its 45-minute timeout.
Evidence cache churn. Evidence is now saved only in three cases: a failed extraction that made progress, a successful owner build, or when nothing was cached yet. A build that keeps failing no longer saves a new cache every 15 minutes.
Owner cache key included sync_project.py. The key now hashes scripts/*.py except sync_project.py, so it still covers blame_snapshot.py and format_maps.py, which build_owners.py imports.

All 137 tests pass. The README and the PR description are updated.

I have not linted this commit or run it live. I'm least sure about two things: the ! exclusion in hashFiles and the hashFiles inside the evidence-save if:. If you rerun your QA Project, those are the parts to watch.

ronishrohan

Approved after checking 8d1e69b.

All 137 Python tests pass. The stateful 512-issue backfill completes across 26 runs within the 100-write limit. All GitHub CI checks pass, including the full frontend suite, Linux browser integration, renderer smoke, Mac helpers and Gitleaks. The privileged owners/sync jobs correctly skip PR runs.

Manual GitHub testing passed 19 scenarios before the final commit and six further checks on this head. Verified bounded writes, stop/resume, interrupted-update recovery, preservation of human edits, close/reopen handling and unchanged reruns. The final capacity check used the real API with only the in-process capacity threshold lowered; existing items updated while new additions were skipped with a nonzero exit.

Proof: private QA Project, final backfill fixtures, and CI run. The QA Project is closed and the fixture repository archived. Production settings and existing issues were untouched.

The narrowed App installation token and hosted Actions cache lifecycle still need their post-merge check; live tests used the authorized CLI OAuth token. Board rotation remains an explicit administrator operation, with no automatic deletion of archive history.

OrcInc added the needs-triage New item has not been fully classified. label Oct 21, 2026

OrcInc added this to the Reviews & workflow milestone Oct 21, 2026

OrcInc added the enhancement New feature or request label Oct 23, 2026

Rishet11 marked this pull request as ready for review October 23, 2026 22:09

Copilot AI balanced review requested due to automatic review settings October 23, 2026 22:09

Copilot AI reviewed Oct 23, 2026

ronishrohan reviewed Oct 24, 2026

View reviewed changes

Comment thread .github/scripts/contributor-project/scripts/sync_project.py Resolved

Comment thread .github/scripts/contributor-project/scripts/sync_project.py Resolved

ronishrohan previously approved these changes Oct 24, 2026

View reviewed changes

ronishrohan and others added 9 commits October 25, 2026 02:27

fix(ci): bound contributor sync and recover interrupted writes

f63283a

fix(ci): reuse contributor ownership evidence between refreshes

36b667b

fix(ci): isolate owner refreshes and preserve extraction progress

a1c448a

fix(ci): account for retry writes and document bounded sync operation

e52492d

chore(ci): group sync summary output for shellcheck

75713bd

fix(ci): keep owner cache keys stable across builds

48a97af

test(ci): prove owner extraction resumes cached API pages

2837e18

fix(ci): restart inconsistent cached ownership pagination

73dd92e

ronishrohan approved these changes Oct 24, 2026

View reviewed changes

Rishet11 merged commit b261f3e into main Oct 24, 2026
7 checks passed

Rishet11 deleted the feat/contributor-project-sync branch October 24, 2026 22:13

pullfrog Bot mentioned this pull request Oct 25, 2026

chore: sync upstream OrchestratorInc/agent-orchestrator@2782e4d7f691 biz-direct/agent-orchestrator#75

Merged

mohakchakraborty2004 mentioned this pull request Oct 27, 2026

release: promote main (c7a99e1ad8) to stable, 2026-10-27 #7314

Merged

Navigation Menu

Repository navigation

feat(ci): sync open issues to a contributor Project with suggested areas and owners - #6995

feat(ci): sync open issues to a contributor Project with suggested areas and owners#6995
Rishet11 merged 14 commits into
mainfrom
feat/contributor-project-sync

Rishet11 commented Oct 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

ronishrohan left a comment

Uh oh!

Uh oh!

Uh oh!

ronishrohan left a comment

Uh oh!

Rishet11 commented Oct 24, 2026

Uh oh!

ronishrohan left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Rishet11 commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Scaling and recovery

Setup after merge (admin)

Testing

Checklist

Uh oh!

github-actions Bot commented Oct 21, 2026

🏆 Review leaderboard

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

ronishrohan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ronishrohan left a comment

Choose a reason for hiding this comment

Uh oh!

Rishet11 commented Oct 24, 2026

Uh oh!

ronishrohan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Rishet11 commented Oct 21, 2026 •

edited

Loading