Skip to content

Commit cb97e0b

Browse files
committed
ci: address the fifth round of review comments
- Look for the commit's Rust (fork) check after waiting for the fork run, in createCheck, so a check created meanwhile is updated instead of duplicated, and drop the earlier lookup in notify() along with its dead fork guard. - Check out the labeler config so actions/labeler reads it from the file. - Treat a 422 from createLabel as success, since another run can create the release label first. - Use a sparse checkout without persisted credentials in the release version labeler, and pin actions/github-script by commit. - List the merge queue cleanup tests with the other CI workflow tests.
1 parent 024e770 commit cb97e0b

7 files changed

Lines changed: 100 additions & 33 deletions

File tree

‎.github/workflows/fork_ci_status.yml‎

Lines changed: 21 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -208,13 +208,30 @@ jobs:
208208
const payload = run
209209
? runPayload(fork, sha, run)
210210
: missingPayload(fork, sha, await workflowState(fork));
211-
await github.rest.checks.create({ ...upstream, name: CHECK, head_sha: sha, ...payload });
212-
core.info(`Created ${CHECK} for ${sha}: ${payload.conclusion || payload.status}`);
211+
// The commit can already have the check. A pull request of the
212+
// same branch to a release branch, or a reopened pull request,
213+
// signals it again, and the scheduled update can create the check
214+
// while this run waits for the fork run. So look for it only now,
215+
// and update it instead of adding a second one.
216+
const { data } = await github.rest.checks.listForRef({
217+
...upstream,
218+
ref: sha,
219+
check_name: CHECK,
220+
per_page: 1,
221+
});
222+
const [existing] = data.check_runs;
223+
if (existing) {
224+
await github.rest.checks.update({ ...upstream, check_run_id: existing.id, ...payload });
225+
} else {
226+
await github.rest.checks.create({ ...upstream, name: CHECK, head_sha: sha, ...payload });
227+
}
228+
core.info(`${existing ? 'Updated' : 'Created'} ${CHECK} for ${sha}: ${payload.conclusion || payload.status}`);
213229
}
214230
215231
async function notify() {
216-
// The same branch can also target a release branch, which gets no
217-
// mirrored check, so look only at the pull request to `main`.
232+
// Only fork PRs to main skip jobs upstream. The job runs only for
233+
// a fork, and a release branch has no merge queue and keeps the
234+
// full run, so look only at the pull request to `main`.
218235
const pulls = await require('./ci/scripts/pull_requests_of_run.js')({
219236
github,
220237
context,
@@ -229,25 +246,6 @@ jobs:
229246
core.info('No open pull request to main at this commit; nothing to mirror.');
230247
return;
231248
}
232-
// Only fork PRs to main skip jobs upstream; release branches have
233-
// no merge queue and keep the full run.
234-
if (!isForkPrToMain(pr)) {
235-
core.info('Not a fork pull request to main; nothing to mirror.');
236-
return;
237-
}
238-
// A pull request of the same branch to a release branch, or a
239-
// reopened pull request, signals a commit that can already have
240-
// the check. Update that check instead of adding a second one.
241-
const { data } = await github.rest.checks.listForRef({
242-
...upstream,
243-
ref: pr.head.sha,
244-
check_name: CHECK,
245-
per_page: 1,
246-
});
247-
if (data.check_runs.length > 0) {
248-
await syncCheck(pr);
249-
return;
250-
}
251249
await createCheck(pr, 12);
252250
}
253251

‎.github/workflows/labeler.yml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,13 @@ jobs:
3030
contents: read
3131
pull-requests: write
3232
steps:
33-
# This checks out the default branch, never the pull request.
33+
# This checks out the default branch, never the pull request. The labeler
34+
# config is checked out too, so the action reads it from the file.
3435
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
3536
with:
36-
sparse-checkout: ci/scripts
37+
sparse-checkout: |
38+
ci/scripts
39+
.github/workflows/labeler
3740
persist-credentials: false
3841

3942
# The same branch can target `main` and a release branch, so label every

‎.github/workflows/release_version_labeler.yaml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,8 +24,11 @@ jobs:
2424
contents: read
2525
steps:
2626
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
27+
with:
28+
sparse-checkout: ci/scripts
29+
persist-credentials: false
2730
- name: Add the target release version
28-
uses: actions/[email protected]
31+
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
2932
with:
3033
script: |
3134
const labeler = require('./ci/scripts/release_version_labeler.js');

‎ci/scripts/release_version_labeler.js‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,15 +36,21 @@ module.exports = async ({ github, context }) => {
3636
name: target_version
3737
});
3838
} catch (error) {
39-
if (error.status === 404) {
39+
if (error.status !== 404) {
40+
throw error;
41+
}
42+
try {
4043
await github.rest.issues.createLabel({
4144
owner: context.repo.owner,
4245
repo: context.repo.repo,
4346
name: target_version,
4447
color: '222222'
4548
});
46-
} else {
47-
throw error;
49+
} catch (error) {
50+
// A run for another push can create the label first.
51+
if (error.status !== 422) {
52+
throw error;
53+
}
4854
}
4955
}
5056

‎ci/scripts/test_fork_ci_status.cjs‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,9 @@ async function execute({
9696
slice = [pr],
9797
openPrs = [pr],
9898
triggeringRun = workflowRun,
99+
// Created by another run, such as the sweep, while this run looks for the
100+
// fork run.
101+
checkCreatedMeanwhile,
99102
calls = [],
100103
}) {
101104
const writes = [];
@@ -114,6 +117,7 @@ async function execute({
114117
actions: {
115118
listWorkflowRuns: async (params) => {
116119
calls.push("actions.listWorkflowRuns");
120+
check = checkCreatedMeanwhile ?? check;
117121
assert.equal(params.owner, "contributor");
118122
assert.equal(params.repo, "datafusion");
119123
assert.equal(params.workflow_id, "rust.yml");
@@ -205,6 +209,17 @@ test("notification updates the check a commit already has instead of adding one"
205209
assertRun(writes[0], "update", latest);
206210
});
207211

212+
test("notification updates a check created while it looked for the fork run", async () => {
213+
const latest = run(2);
214+
const writes = await execute({
215+
event: "workflow_run",
216+
checkCreatedMeanwhile: checkFor(run(1)),
217+
runs: [latest],
218+
});
219+
assert.equal(writes.length, 1);
220+
assertRun(writes[0], "update", latest);
221+
});
222+
208223
test("notification skips a commit that is no longer a pull request head", async () => {
209224
const writes = await execute({
210225
event: "workflow_run",

‎ci/scripts/test_release_version_labeler.cjs‎

Lines changed: 43 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,21 +19,36 @@ const pull = (
1919

2020
// `lookups` maps a commit to the pull requests returned on each attempt (the
2121
// last response repeats once the attempts run out), or to an error to throw.
22-
// `failLabels` lists the pull requests that fail to get the label.
23-
async function run({ lookups, commits, failLabels = [] }) {
22+
// `failLabels` lists the pull requests that fail to get the label. `label` is
23+
// "missing" when the release label does not exist, or "raced" when another run
24+
// creates it between the lookup and the creation.
25+
async function run({ lookups, commits, failLabels = [], label = "exists" }) {
2426
const result = {
2527
listed: [],
2628
labeled: [],
2729
delays: [],
2830
logs: [],
2931
inFlight: 0,
3032
maxInFlight: 0,
33+
createdLabels: 0,
3134
};
3235
const attempts = {};
3336
const github = {
3437
rest: {
3538
issues: {
36-
getLabel: async () => {},
39+
getLabel: async () => {
40+
if (label !== "exists") {
41+
throw Object.assign(new Error("Not Found"), { status: 404 });
42+
}
43+
},
44+
createLabel: async () => {
45+
if (label === "raced") {
46+
throw Object.assign(new Error("Validation Failed"), {
47+
status: 422,
48+
});
49+
}
50+
result.createdLabels++;
51+
},
3752
addLabels: async ({ issue_number, labels }) => {
3853
if (failLabels.includes(issue_number)) {
3954
throw new Error(`cannot label ${issue_number}`);
@@ -213,3 +228,28 @@ test("logs every failed request with its commit or pull request", async () => {
213228
"2 request(s) failed, see above.",
214229
]);
215230
});
231+
232+
test("creates the release label when it is missing", async () => {
233+
const result = await execute({
234+
commits: ["a"],
235+
lookups: { a: [[pull(1)]] },
236+
label: "missing",
237+
});
238+
assert.equal(result.createdLabels, 1);
239+
assert.deepEqual(
240+
result.labeled.map(([number]) => number),
241+
[1]
242+
);
243+
});
244+
245+
test("labels when another run creates the release label first", async () => {
246+
const result = await execute({
247+
commits: ["a"],
248+
lookups: { a: [[pull(1)]] },
249+
label: "raced",
250+
});
251+
assert.deepEqual(
252+
result.labeled.map(([number]) => number),
253+
[1]
254+
);
255+
});

‎docs/source/contributor-guide/testing_infrastructure.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,17 +10,19 @@ This page covers checks for CI workflows, documentation, repository maintenance,
1010

1111
## CI Workflow Tests
1212

13-
The [Rust workflow validator tests], [fork status bridge tests] and [release version labeler tests] run in CI and in `./dev/rust_lint.sh`.
13+
The [Rust workflow validator tests], [fork status bridge tests], [merge queue cleanup tests] and [release version labeler tests] run in CI and in `./dev/rust_lint.sh`.
1414
Run them locally:
1515

1616
```shell
1717
python3 ci/scripts/test_rust_workflow_skips.py
1818
node --test ci/scripts/test_fork_ci_status.cjs
19+
node --test ci/scripts/test_cancel_merge_queue_runs.cjs
1920
node --test ci/scripts/test_release_version_labeler.cjs
2021
```
2122

2223
[rust workflow validator tests]: https://github.com/apache/datafusion/blob/main/ci/scripts/test_rust_workflow_skips.py
2324
[fork status bridge tests]: https://github.com/apache/datafusion/blob/main/ci/scripts/test_fork_ci_status.cjs
25+
[merge queue cleanup tests]: https://github.com/apache/datafusion/blob/main/ci/scripts/test_cancel_merge_queue_runs.cjs
2426
[release version labeler tests]: https://github.com/apache/datafusion/blob/main/ci/scripts/test_release_version_labeler.cjs
2527

2628
## Documentation Examples

0 commit comments

Comments
 (0)