Skip to content

[chore](ci) Connect PR checks and comment triggers to current TeamCity - #70459

Merged
hello-stephen merged 7 commits into
apache:masterfrom
hello-stephen:codex/teamcity-pr-onboarding-20261111
Nov 11, 2026
Merged

hello-stephen merged 7 commits into
apache:masterfrom
hello-stephen:codex/teamcity-pr-onboarding-20261111

Conversation

@hello-stephen

@hello-stephen hello-stephen commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Background

Doris PR comments use the existing workflow and shell helper to select and trigger TeamCity pipelines. This migration replaces the retired environment and embedded credentials with the current TeamCity and a dedicated token.

What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Connect master, branch-4.2 and branch-4.1 PR checks and comment commands to the current TeamCity. Preserve file-based trigger/skip selection and authorized skip buildall. Keep the comment entry point simple: submit the requested pipeline with PR number, target branch and an exact VCS revision; artifact resolution remains the target pipeline's responsibility.

Details

  • Change map: The existing shell helper is 210 lines; tests cover direct submission, command parsing, file filters, skip status coverage, authorization and credential tracing. The workflow retains the original file-selection functions; .asf.yaml updates required contexts for three branches. No separate production Python client or new dependencies.
  • Affected: Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0 and NonConcurrent remain required. External and Performance are optional. Current command keywords match the TeamCity/<keyword> status suffix.
  • Developer and release impact: run p0 submits P0 directly, with only env.PR_NUMBER and env.CI_BRANCH as business parameters and the commit in TeamCity's native revision field. The client does not search or convert artifact manifests, choose producers, automatically compile missing artifacts, override artifact dependencies or deduplicate requests. TeamCity owns artifact retrieval and queue replacement. run buildall and the existing authorized skip buildall policy remain; Performance is enabled only for master using a positive branch condition. Temporary validation VCS roots are rejected before submission.
  • File selection: Existing FEUT, BEUT, CloudUT, regression and Performance file-pattern functions remain unchanged. trigger_or_skip_build submits related changes and publishes skip statuses for unrelated changes. Skipping Compile also resolves its five downstream checks. Failed file lookups do not falsely select a skip.
  • Behavior change: The new environment publishes current check names; External and Performance no longer block merges. Commands run trusted default-branch code. Token authentication replaces the retired endpoint and public account credentials; authenticated requests are not traced.
  • License and third-party: Existing Apache licensing; no dependency additions.
  • Rollback: Revert workflow routing and required-context changes together without restoring retired credentials or interrupting running builds.

Testing

  • Local: 26 tests pass, including direct submission of all five regression lanes with only two business parameters; exact revision binding; repeated submissions; file filtering; skip buildall permissions and status coverage on master and both release branches; stale heads; HTTP failures; and token redaction.
  • Local: actionlint, shell syntax validation and git diff --check pass.
  • Live: Previous CloudUT submission through the existing shell helper completed successfully on branch-4.2 (764 passed, 27 ignored). Earlier Compile canaries completed on all three branches. These results predate this final simplification.
  • Live: Final direct P0 submission 1065 retained the exact requested commit and did not create a Compile dependency. It failed before execution because the target pipeline still selects build number UNBOUND; commit-based artifact lookup is not implemented there yet. Compile 852 has an unexpired artifact manifest for this exact PR/head, so this failure is a pipeline lookup gap rather than missing compilation. All five regression configurations currently use the same UNBOUND default. CloudUT canaries 1066 (master), 1067 (branch-4.2), and 1068 (branch-4.1) are running. The live harness uses the committed shell helper and substitutes server-local authentication/PR-read transport; it does not post GitHub comments.

Notes for Reviewers

The issue_comment workflow becomes active only after merge into the default branch. Live branch protection must be checked after ASF reconciliation. Regression pipelines must resolve artifacts for the requested commit and fail when unavailable; the comment client deliberately has no producer fallback. The experimental commit-tag approach is not part of this PR and has not been deployed to production regression configurations. Performance's temporary validation VCS configuration still prevents general PR comment runs.

Release note

None.

Check List (For Author)

  • Behavior changed: Yes (see Details)
  • Does this need documentation: No
  • Pre-submission status: Partially verified: local checks passed; P0 commit-based artifact lookup remains blocked by the server configuration; CloudUT canaries, post-merge comment execution and branch protection reconciliation remain pending.

AI Disclosure

  • Tool: Codex
  • Generated: Workflow, shell helper changes, tests and this description.
  • Human review: Pending.

### Background

Doris pull requests use required GitHub status contexts defined in `.asf.yaml`. The migrated TeamCity configurations publish their results under `TeamCity/...` names.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Move master, branch-4.2, and branch-4.1 to the eight status contexts published by the current TeamCity pipelines. External and Performance remain visible CI results but are optional while their environments are being repaired.

### Details

- **Affected:** Required status checks for master, branch-4.2, and branch-4.1 only. Keep Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0, and NonConcurrent required under their current names. Remove External from all three branches and Performance from master; Performance is already optional on both release branches.
- **Developer and release impact:** Pull requests wait for the current pipelines. External and Performance failures do not prevent merging. Review requirements and other checks remain unchanged.
- **License and third-party:** None.
- **Rollback:** Revert this configuration change together with restoring the previous CI entry point.

### Testing

Parsed both YAML versions and compared their structure. Verified all eight new contexts per target branch, removal of optional contexts, unchanged review rules and unrelated checks, and byte-equivalent configuration data outside the three target branches. `git diff --check` passes. Live PR canaries are in progress; no full pipeline success is claimed.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: configuration validation passed; live pipeline canaries and post-merge branch-protection reconciliation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Configuration changes and this description.
- Human review: Pending.
### Background

Doris pull requests use required GitHub status contexts defined in `.asf.yaml`. The migrated TeamCity configurations publish their results under `TeamCity/...` names.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Move master, branch-4.2, and branch-4.1 to the eight status contexts published by the current TeamCity pipelines. External and Performance remain visible CI results but are optional while their environments are being repaired.

### Details

- **Affected:** Required status checks for master, branch-4.2, and branch-4.1 only. Keep Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0, and NonConcurrent required under their current names. Remove External from all three branches and Performance from master; Performance is already optional on both release branches.
- **Developer and release impact:** Pull requests wait for the current pipelines. External and Performance failures do not prevent merging. Review requirements and other checks remain unchanged.
- **License and third-party:** None.
- **Rollback:** Revert this configuration change together with restoring the previous CI entry point.

### Testing

Parsed both YAML versions and compared their structure. Verified all eight new contexts per target branch, removal of optional contexts, unchanged review rules and unrelated checks, and byte-equivalent configuration data outside the three target branches. `git diff --check` passes. Live PR canaries are in progress; no full pipeline success is claimed.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: configuration validation passed; live pipeline canaries and post-merge branch-protection reconciliation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Configuration changes and this description.
- Human review: Pending.
…tials

### Background

Doris PR comments currently submit builds to the legacy TeamCity using credentials embedded in a shared shell helper. The migrated TeamCity publishes `TeamCity/...` statuses and requires regression consumers to use artifacts from the exact successful Compile build.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Connect master, branch-4.2, and branch-4.1 required checks and PR comment commands to the current TeamCity. Keep External and Performance optional, preserve active builds, and bind individual regression reruns to verified artifacts from the requested PR revision. Remove embedded legacy credentials and authenticated command tracing; current triggers use a separate project-scoped account stored in GitHub Actions Secrets.

### Details

- **Change map:** About 510 lines implement the current comment client, 217 lines test command safety and artifact binding, and about 100 lines adjust workflow routing and credential handling; `.asf.yaml` changes the three branches' required contexts. No generated files or dependency additions.
- **Affected:** Eight required contexts per migrated branch: Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0, and NonConcurrent. Current comment commands use pinned PR revisions, reconcile duplicate requests, preserve active builds, and either reuse a terminal verified Compile artifact or queue its producer. Other branches retain their existing pipeline mapping with credentials supplied by Secrets.
- **Developer and release impact:** `run buildall`, individual supported lanes, and the existing authorized skip policy use current status names on the migrated branches. Comment triggers require repository write access. P1, CloudP1, and coverage have no migrated configurations and cannot silently trigger the legacy server. Performance uses a separate verified Release producer and master baseline; a temporary validation VCS root is rejected before queueing, so it needs its general PR configuration restored before universal comment reruns work.
- **Behavior change:** External and Performance do not gate merges. Running jobs are reused and never canceled by the current comment client. Commands execute only from the default branch, not PR-provided workflow code.
- **Configs and defaults:** `TEAMCITY_CURRENT_URL`, `TEAMCITY_CURRENT_TOKEN`, `TEAMCITY_LEGACY_USER`, and `TEAMCITY_LEGACY_PASSWORD` have been provisioned. Current credentials grant only the Doris project permissions needed to inspect and submit builds; no cancellation, configuration editing, or server administration. Moving the already-public legacy credential into Secrets does not revoke its historical exposure; rotation on the legacy server remains outstanding.
- **License and third-party:** Apache license headers; Python standard library only.
- **Rollback:** Revert workflow routing and required-context changes together. Preserve in-flight builds and coordinate any credential rollback with remaining legacy consumers.

### Testing

- Local: 17 contract tests pass, covering PR updates, repeat counts, request reconciliation, active-build reuse, stale/modified artifacts, native artifact dependency pinning, authorization, temporary VCS root rejection, and credential suppression under shell tracing.
- Local: `actionlint`, shell syntax validation, YAML parsing, and `git diff --check` pass.
- Live: The dedicated account can read required project data and submit an exact-revision CloudUT build; a repeat submission reused the same build. Global server settings return 403. The client validated a real successful Compile artifact and all nine general PR configuration roots.
- Live: Six CloudUT canaries across two PRs on each target branch finished successfully. The additional comment-client probe exposed a bootstrap checksum mismatch after the dependency-cache update; the checksums were corrected and build 933 is validating the repair. Remaining Compile, UT, and regression canaries are in progress; this is not a full-green claim.

### Notes for Reviewers

The `issue_comment` workflow only becomes active after merge into the default branch. Live GitHub branch protection must be read back after ASF configuration reconciliation; branch-4.1 currently has no effective protection despite the source configuration. No live GitHub test comment was posted. Performance's temporary validation configuration is deliberately rejected rather than reused for unrelated PRs.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: local checks and live API submission passed; full canaries, post-merge issue-comment execution, branch-protection reconciliation, and legacy credential rotation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Configuration, comment client, tests, and this description.
- Human review: Pending.
@hello-stephen hello-stephen changed the title [chore](ci) Connect required checks to current TeamCity pipelines [chore](ci) Connect PR checks and comment triggers to current TeamCity Nov 11, 2026
### Background

Doris PR comments currently submit builds to the legacy TeamCity using credentials embedded in a shared shell helper. The migrated TeamCity publishes `TeamCity/...` statuses and requires regression consumers to use artifacts from the exact successful Compile build.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Connect master, branch-4.2, and branch-4.1 required checks and PR comment commands to the current TeamCity. Keep External and Performance optional, preserve active builds, and bind individual regression reruns to verified artifacts from the requested PR revision. Remove embedded legacy credentials and authenticated command tracing; current triggers use a separate project-scoped account stored in GitHub Actions Secrets.

### Details

- **Change map:** About 300 lines extend the existing `teamcity-utils.sh` with current-server submission and artifact binding, 252 lines test its existing shell entry points, and the existing workflow adds branch routing, authorization and credential handling; `.asf.yaml` changes the three branches' required contexts. No generated files or dependency additions.
- **Reuse:** Keep the existing comment parser, workflow job and `trigger_or_skip_build` / `trigger_build` APIs. Extend the existing revision-aware queue lookup and request helper; there is no separate production Python entry point. Inline Python only validates and converts artifact manifests.
- **Affected:** Eight required contexts per migrated branch: Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0, and NonConcurrent. Current comment commands use pinned PR revisions, reconcile duplicate requests, preserve active builds, and either reuse a terminal verified Compile artifact or queue its producer. Other branches retain their existing pipeline mapping with credentials supplied by Secrets.
- **Developer and release impact:** `run buildall`, individual supported lanes, and the existing authorized skip policy use current status names on the migrated branches. Comment triggers require repository write access. P1, CloudP1, and coverage have no migrated configurations and cannot silently trigger the legacy server. Performance uses a separate verified Release producer and master baseline; a temporary validation VCS root is rejected before queueing, so it needs its general PR configuration restored before universal comment reruns work.
- **Behavior change:** External and Performance do not gate merges. Running jobs are reused and never canceled by the current comment client. Commands execute only from the default branch, not PR-provided workflow code.
- **Configs and defaults:** `TEAMCITY_CURRENT_URL`, `TEAMCITY_CURRENT_TOKEN`, `TEAMCITY_LEGACY_USER`, and `TEAMCITY_LEGACY_PASSWORD` have been provisioned. Current credentials grant only the Doris project permissions needed to inspect and submit builds; no cancellation, configuration editing, or server administration. Moving the already-public legacy credential into Secrets does not revoke its historical exposure; rotation on the legacy server remains outstanding.
- **License and third-party:** Apache license headers; Python standard library only.
- **Rollback:** Revert workflow routing and required-context changes together. Preserve in-flight builds and coordinate any credential rollback with remaining legacy consumers.

### Testing

- Local: 19 shell-entry-point tests pass, covering PR updates, repeat counts, request reconciliation, active-build reuse, stale/modified artifacts, native artifact dependency pinning, authorization, temporary VCS root rejection, and credential suppression under shell tracing.
- Local: `actionlint`, shell syntax validation, YAML parsing, and `git diff --check` pass.
- Live: The dedicated account can read required project data and submit an exact-revision CloudUT build; a repeat submission reused the same build. Global server settings return 403. The client validated a real successful Compile artifact and all nine general PR configuration roots.
- Live: The refactored shell helper reused running Compile, BEUT and P0 builds without new submissions. It then submitted CloudUT build 974 for branch-4.2; replaying the same request returned 974 without a second submission. This refactor probe used server-side test authorization; the dedicated account permissions were verified separately as described above. Build 974 is still running.
- Live: Six CloudUT canaries across two PRs on each target branch finished successfully. The additional comment-client probe exposed a bootstrap checksum mismatch after the dependency-cache update; the checksums were corrected and build 933 finished successfully (858 passed, 27 ignored, no failures). A successful master Compile fanned out all five regression lanes with matching artifact bindings. Master BEUT also finished successfully (16,478 passed, 115 ignored), and VaultP0 passed all 21 tests. The checkout-repair CloudUT retry finished successfully (858 passed, 27 ignored). Other Compile, UT, and regression canaries are in progress; this is not a full-green claim.

### Notes for Reviewers

The `issue_comment` workflow only becomes active after merge into the default branch. Live GitHub branch protection must be read back after ASF configuration reconciliation; branch-4.1 currently has no effective protection despite the source configuration. No live GitHub test comment was posted. Performance's temporary validation configuration is deliberately rejected rather than reused for unrelated PRs.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: local checks and live API submission passed; full canaries, post-merge issue-comment execution, branch-protection reconciliation, and legacy credential rotation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Configuration, existing shell helper extensions, tests, and this description.
- Human review: Pending.
…ands

### Background

Doris PR comments currently submit builds to the legacy TeamCity using credentials embedded in a shared shell helper. The migrated TeamCity publishes `TeamCity/...` statuses and requires regression consumers to use artifacts from the exact successful Compile build.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Connect master, branch-4.2, and branch-4.1 required checks and PR comment commands to the current TeamCity. Keep External and Performance optional, preserve active builds, and bind individual regression reruns to verified artifacts from the requested PR revision. Remove embedded legacy credentials and authenticated command tracing; current triggers use a separate project-scoped account stored in GitHub Actions Secrets.

### Details

- **Change map:** About 300 lines extend the existing `teamcity-utils.sh` with current-server submission and artifact binding, 318 lines test its existing shell entry points and workflow parser, and the existing workflow adds branch routing, authorization and credential handling; `.asf.yaml` changes the three branches' required contexts. No generated files or dependency additions.
- **Reuse:** Keep the existing comment parser, workflow job and `trigger_or_skip_build` / `trigger_build` APIs. Extend the existing revision-aware queue lookup and request helper; there is no separate production Python entry point. Inline Python only validates and converts artifact manifests.
- **Affected:** Eight required contexts per migrated branch: Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0, and NonConcurrent. Comment keywords are checked against all `TeamCity/...` contexts in `.asf.yaml`, plus optional External and Performance. Current comment commands use pinned PR revisions, reconcile duplicate requests, preserve active builds, and either reuse a terminal verified Compile artifact or queue its producer. Only master, branch-4.2 and branch-4.1 are enabled by this migration; other branches are rejected without contacting a retired server.
- **Developer and release impact:** `run <keyword>` matches the exact `TeamCity/<keyword>` status suffix, including lowercase `run nonconcurrent`. `run buildall` remains the aggregate command, and the existing authorized skip policy uses the same current status names. Comment triggers require repository write access. P1, CloudP1, and coverage commands are removed because no corresponding current configurations exist. Performance explicitly requires `TARGET_BRANCH == 'master'`; release-branch buildall does not include it. Performance uses a separate verified Release producer and master baseline; a temporary validation VCS root is rejected before queueing, so it needs its general PR configuration restored before universal comment reruns work.
- **Behavior change:** External and Performance do not gate merges. Running jobs are reused and never canceled by the current comment client. Commands execute only from the default branch, not PR-provided workflow code.
- **Configs and defaults:** `TEAMCITY_CURRENT_URL` and `TEAMCITY_CURRENT_TOKEN` are provisioned. The retired server URL, Basic-auth path and legacy build IDs have been removed; both temporary legacy GitHub Secrets have been deleted. Current credentials grant only the Doris project permissions needed to inspect and submit builds; no cancellation, configuration editing, or server administration. The old environment no longer exists; this workflow has no legacy credential or endpoint fallback.
- **License and third-party:** Apache license headers; Python standard library only.
- **Rollback:** Revert workflow routing and required-context changes together. Preserve in-flight builds; do not restore the retired server or its public credentials.

### Testing

- Local: 25 shell-entry-point and workflow-parser tests pass, covering PR updates, repeat counts, request reconciliation, active-build reuse, stale/modified artifacts, native artifact dependency pinning, authorization, temporary VCS root rejection, and credential suppression under shell tracing.
- Local: `actionlint`, shell syntax validation, YAML parsing, and `git diff --check` pass.
- Live: The dedicated account can read required project data and submit an exact-revision CloudUT build; a repeat submission reused the same build. Global server settings return 403. The client validated a real successful Compile artifact and all nine general PR configuration roots.
- Live: The refactored shell helper reused running Compile, BEUT and P0 builds without new submissions. It then submitted CloudUT build 974 for branch-4.2; replaying the same request returned 974 without a second submission. This refactor probe used server-side test authorization; the dedicated account permissions were verified separately as described above. Build 974 finished successfully (764 passed, 27 ignored, no failures).
- Live: Six CloudUT canaries across two PRs on each target branch finished successfully. The additional comment-client probe exposed a bootstrap checksum mismatch after the dependency-cache update; the checksums were corrected and build 933 finished successfully (858 passed, 27 ignored, no failures). A successful master Compile fanned out all five regression lanes with matching artifact bindings. Master BEUT also finished successfully (16,478 passed, 115 ignored), and VaultP0 passed all 21 tests. The checkout-repair CloudUT retry finished successfully (858 passed, 27 ignored). Compile completed successfully on all three branches. Remaining canaries include both running builds and failures requiring further classification; this is not a full-green claim.

### Notes for Reviewers

The `issue_comment` workflow only becomes active after merge into the default branch. Live GitHub branch protection must be read back after ASF configuration reconciliation; branch-4.1 currently has no effective protection despite the source configuration. No live GitHub test comment was posted. Performance's temporary validation configuration is deliberately rejected rather than reused for unrelated PRs.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: local checks and live API submission passed; full canaries, post-merge issue-comment execution and branch-protection reconciliation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Configuration, existing shell helper extensions, tests, and this description.
- Human review: Pending.
…e deduplication

### Background

Doris PR comments currently submit builds to the legacy TeamCity using credentials embedded in a shared shell helper. The migrated TeamCity publishes `TeamCity/...` statuses and requires regression consumers to use artifacts from the exact successful Compile build.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Connect master, branch-4.2, and branch-4.1 required checks and PR comment commands to the current TeamCity. Keep External and Performance optional, preserve the existing per-file trigger/skip decisions, and bind individual regression reruns to verified artifacts from the requested PR revision. Remove embedded legacy credentials and authenticated command tracing; current triggers use a separate project-scoped account stored in GitHub Actions Secrets.

### Details

- **Change map:** The final shell helper is 355 lines, including current-server submission and artifact binding; 398 lines of tests cover its entry points and workflow parser/file-filter wiring. The workflow retains the original file-selection functions while adding branch routing, authorization and credential handling; `.asf.yaml` changes the three branches' required contexts. No generated files or dependency additions.
- **Reuse:** Keep the existing comment parser, workflow job and `trigger_or_skip_build` / `trigger_build` APIs. Reuse the existing `github-utils.sh` file-selection functions and trigger/skip API; there is no separate production Python entry point. Inline Python only validates and converts artifact manifests.
- **Affected:** Eight required contexts per migrated branch: Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0, and NonConcurrent. Comment keywords are checked against all `TeamCity/...` contexts in `.asf.yaml`, plus optional External and Performance. Current comment commands use pinned PR revisions and either reuse a terminal verified Compile artifact or queue its producer. Duplicate submission and queue replacement are delegated to TeamCity; there is no client-side history/queue deduplication, request key, or GitHub concurrency gate. Only master, branch-4.2 and branch-4.1 are enabled by this migration; other branches are rejected without contacting a retired server.
- **File selection:** The workflow passes the original FEUT, BEUT, CloudUT, regression and Performance file-filter results into `trigger_or_skip_build`. Related changes submit a build; unrelated changes publish the corresponding skip status. Compile runs when either P0 or CloudP0 requires it; skipping Compile also resolves P0, CloudP0, VaultP0, External and NonConcurrent checks. Existing file-pattern rules remain unchanged. A failed file lookup never selects a skip; the pipeline-script check still requires a valid file list.
- **Developer and release impact:** `run <keyword>` matches the exact `TeamCity/<keyword>` status suffix, including lowercase `run nonconcurrent`. `run buildall` remains the aggregate command, and the existing authorized skip policy uses the same current status names. Comment triggers require repository write access. P1, CloudP1, and coverage commands are removed because no corresponding current configurations exist. Performance explicitly requires `TARGET_BRANCH == 'master'`; release-branch buildall does not include it. Performance uses a separate verified Release producer and master baseline; a temporary validation VCS root is rejected before queueing, so it needs its general PR configuration restored before universal comment reruns work.
- **Behavior change:** External and Performance do not gate merges. The client submits each requested run to TeamCity without looking up or canceling duplicate builds. Skip status writes verify the PR head and report HTTP errors. Commands execute only from the default branch, not PR-provided workflow code.
- **Configs and defaults:** `TEAMCITY_CURRENT_URL` and `TEAMCITY_CURRENT_TOKEN` are provisioned. The retired server URL, Basic-auth path and legacy build IDs have been removed; both temporary legacy GitHub Secrets have been deleted. Current credentials grant only the Doris project permissions needed to inspect and submit builds; no cancellation, configuration editing, or server administration. The old environment no longer exists; this workflow has no legacy credential or endpoint fallback.
- **License and third-party:** Apache license headers; Python standard library only.
- **Rollback:** Revert workflow routing and required-context changes together. Preserve in-flight builds; do not restore the retired server or its public credentials.

### Testing

- Local: 29 shell-entry-point and workflow-parser tests pass, covering the original file filters with FE, Cloud, README and regression-config fixtures; workflow flag wiring; Compile/downstream skip statuses; failed skip writes and stale PR heads; submission of repeated requests to TeamCity; native artifact binding; authorization; branch restrictions; and suppression of authenticated request tracing.
- Local: `actionlint`, shell syntax validation, YAML parsing, and `git diff --check` pass.
- Live: The dedicated account can read required project data and submit an exact-revision CloudUT build. Global server settings return 403. The client validated a real successful Compile artifact and all nine general PR configuration roots.
- Live: Shell submission probe 974 for branch-4.2 finished successfully (764 passed, 27 ignored, no failures). The final file-filter and server-owned-dedup changes are validated locally; no live GitHub test comment has been posted.
- Live: Six CloudUT canaries across two PRs on each target branch finished successfully. The additional comment-client probe exposed a bootstrap checksum mismatch after the dependency-cache update; the checksums were corrected and build 933 finished successfully (858 passed, 27 ignored, no failures). A successful master Compile fanned out all five regression lanes with matching artifact bindings. Master BEUT also finished successfully (16,478 passed, 115 ignored), and VaultP0 passed all 21 tests. The checkout-repair CloudUT retry finished successfully (858 passed, 27 ignored). Compile completed successfully on all three branches. Remaining canaries include both running builds and failures requiring further classification; this is not a full-green claim.

### Notes for Reviewers

The `issue_comment` workflow only becomes active after merge into the default branch. Live GitHub branch protection must be read back after ASF configuration reconciliation; at the initial rollout inspection, branch-4.1 had no effective protection despite the source configuration. No live GitHub test comment was posted. Performance's temporary validation configuration is deliberately rejected rather than reused for unrelated PRs.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: local checks and live API submission passed; full canaries, post-merge issue-comment execution and branch-protection reconciliation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Configuration, existing shell helper extensions, tests, and this description.
- Human review: Pending.
### Background

Doris PR comments use the existing workflow and shell helper to select and trigger TeamCity pipelines. This migration replaces the retired environment and embedded credentials with the current TeamCity and a dedicated token.

### What problem does this PR solve?

Issue Number: None
Related PR: None
PR Type: Build/CI/Chore

Problem Summary:

Connect master, branch-4.2 and branch-4.1 PR checks and comment commands to the current TeamCity. Preserve file-based trigger/skip selection and authorized `skip buildall`. Keep the comment entry point simple: submit the requested pipeline with PR number, target branch and an exact VCS revision; artifact resolution remains the target pipeline's responsibility.

### Details

- **Change map:** The existing shell helper is 210 lines; tests cover direct submission, command parsing, file filters, skip status coverage, authorization and credential tracing. The workflow retains the original file-selection functions; `.asf.yaml` updates required contexts for three branches. No separate production Python client or new dependencies.
- **Affected:** Compile, FEUT, BEUT, CloudUT, P0, CloudP0, VaultP0 and NonConcurrent remain required. External and Performance are optional. Current command keywords match the `TeamCity/<keyword>` status suffix.
- **Developer and release impact:** `run p0` submits P0 directly, with only `env.PR_NUMBER` and `env.CI_BRANCH` as business parameters and the commit in TeamCity's native revision field. The client does not search or convert artifact manifests, choose producers, automatically compile missing artifacts, override artifact dependencies or deduplicate requests. TeamCity owns artifact retrieval and queue replacement. `run buildall` and the existing authorized `skip buildall` policy remain; Performance is enabled only for master using a positive branch condition. Temporary validation VCS roots are rejected before submission.
- **File selection:** Existing FEUT, BEUT, CloudUT, regression and Performance file-pattern functions remain unchanged. `trigger_or_skip_build` submits related changes and publishes skip statuses for unrelated changes. Skipping Compile also resolves its five downstream checks. Failed file lookups do not falsely select a skip.
- **Behavior change:** The new environment publishes current check names; External and Performance no longer block merges. Commands run trusted default-branch code. Token authentication replaces the retired endpoint and public account credentials; authenticated requests are not traced.
- **License and third-party:** Existing Apache licensing; no dependency additions.
- **Rollback:** Revert workflow routing and required-context changes together without restoring retired credentials or interrupting running builds.

### Testing

- Local: 26 tests pass, including direct submission of all five regression lanes with only two business parameters; exact revision binding; repeated submissions; file filtering; skip buildall permissions and status coverage on master and both release branches; stale heads; HTTP failures; and token redaction.
- Local: actionlint, shell syntax validation and git diff --check pass.
- Live: Previous CloudUT submission through the existing shell helper completed successfully on branch-4.2 (764 passed, 27 ignored). Earlier Compile canaries completed on all three branches. These results predate this final simplification.
- Live: Final direct-submission canaries are being rerun after this update; results will be added here. No GitHub test comment has been posted.

### Notes for Reviewers

The issue_comment workflow becomes active only after merge into the default branch. Live branch protection must be checked after ASF reconciliation. Regression pipelines must resolve artifacts for the requested commit and fail when unavailable; the comment client deliberately has no producer fallback. The experimental commit-tag approach is not part of this PR and has not been deployed to production regression configurations. Performance's temporary validation VCS configuration still prevents general PR comment runs.

### Release note

None.

### Check List (For Author)

- Behavior changed: Yes (see Details)
- Does this need documentation: No
- Pre-submission status: Partially verified: local checks passed; final live canaries, post-merge comment execution and branch protection reconciliation remain pending.

### AI Disclosure

- Tool: Codex
- Generated: Workflow, shell helper changes, tests and this description.
- Human review: Pending.
@hello-stephen
hello-stephen merged commit 1f241cf into apache:master Nov 11, 2026
52 of 58 checks passed
@hello-stephen
hello-stephen deleted the codex/teamcity-pr-onboarding-20261111 branch November 11, 2026 08:39
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Nov 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

hello-stephen pushed a commit that referenced this pull request Nov 14, 2026
…h-4.1 (#70578)

### Background

The required status checks of every protected branch come from
`protected_branches` in the `.asf.yaml` on `master`. ASF's asfyaml
service applies that section to GitHub whenever the `github:` block
changes; the copies on release branches have no effect. Since #70459,
PRs into `master`, `branch-4.2` and `branch-4.1` are checked by the new
TeamCity, which reports statuses named `TeamCity/<pipeline>`. External
is the regression lane for external catalogs and their docker fixtures.

### What problem does this PR solve?

Issue Number: None
Related PR: #70459 (introduced the behavior this PR changes), #70457
(must be picked to branch-4.2 and branch-4.1 first)
PR Type: Build/CI/Chore

Problem Summary:

When #70459 renamed the required checks, it left `TeamCity/external`
out, so a red External run no longer blocks merges into `master`,
`branch-4.2` or `branch-4.1`. This PR adds it back to all three lists,
in the slot the old `External Regression (Doris External Regression)`
check had. Merging also makes asfyaml re-apply the whole section, and
GitHub is out of sync with it today: `branch-4.2` still requires the
nine pre-migration names, which new PRs never receive, and `branch-4.1`
and `branch-2.0` have no protection at all.

### Details

- **Affected:** GitHub branch protection only: one more required context
on `master` (17 in total), `branch-4.2` (13) and `branch-4.1` (13). No
workflow, script or TeamCity change: External uses P0's file filter, so
a PR that skips P0 also gets a skipped `TeamCity/external`. Trailing
spaces are dropped from two `branch-4.0` and `branch-3.1` lines, which
changes no value.
- **Developer and release impact:** PRs into the three branches need
`TeamCity/external` green or skipped before they can merge. On
`branch-4.2` and `branch-4.1` its preflight fails today with `Native
rootless support missing; cherry-pick PR70457 before validating this
branch`, so #70457 has to be picked to both branches before this lands,
or every 4.x PR that runs External is blocked. Once protection is
re-applied, `branch-4.1` PRs again need an approval and green required
checks, and `branch-4.2` PRs can merge on the `TeamCity/*` checks they
actually receive.
- **License and third-party:** None.
- **Rollback:** Revert this commit; asfyaml applies the previous lists
again because the `github:` block changes.

### Testing

- `asfyaml-validate` from apache/infrastructure-asfyaml (`e4c2644`), run
locally: the old and the new file are valid, and copies with a
non-integer review count or a mis-indented list item are rejected.
- Both files parsed locally with asfyaml's GitHub schema: the only data
change is `TeamCity/external` in the three `contexts` lists, with the
existing order kept and no duplicates.
- Every name in the new lists is reported on live PRs: all 17 on #70366
(`master`), and all 13 on a recent `branch-4.1` PR.

### Notes for Reviewers

- asfyaml runs only on `master`, after the merge, so whether GitHub
takes the new lists can only be checked then: `gh api
repos/apache/doris/branches/<branch> --jq
.protection.required_status_checks.contexts` for each branch, and
`.protected` for `branch-4.1` and `branch-2.0`. A failed run is mailed
to [email protected] and to whoever merged.
- `TeamCity/performance` stays optional; the comment above the lists now
says only that.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants