Repository navigation
Conversation
….4 migration note with branch-4.x The branch-4.x backport (apache#58438) ships spark.sql.hive.thriftServer.allowIneffectiveDoAs in 4.4.0, as a warn-only gate. Version the conf 4.4.0 here too, since that is the first release with the key, and add the backport's 4.4 migration note to the 4.3 to 4.4 section, which is otherwise identical between master and branch-4.x. Co-authored-by: Isaac <[email protected]>
|
The AI review requested by @HyukjinKwon did not finish. A committer can add the |
spark-test
left a comment
There was a problem hiding this comment.
Review summary
No defects were found in either edit. In StaticSQLConf, spark.sql.hive.thriftServer.allowIneffectiveDoAs now declares .version("4.4.0"), the same as the key's definition on branch-4.x, whose project version is 4.4.0-SNAPSHOT. That follows the repository rule that a conf's version names the release it first ships in. Only the reported since-version changes: the false default, the conf's doc text, and master's refuse-to-start check in HiveThriftServer2.failIfIneffectiveDoAs are unchanged.
With the new bullet, master's "Upgrading from Spark SQL 4.3 to 4.4" section is byte-identical to the same section on the local origin/branch-4.x ref (at b3a476d); before this PR, that bullet was the only difference. The bullet describes the 4.4 code correctly: on that branch, warnIfIneffectiveDoAs runs from the server's init and warns under the same doAs/authentication condition that master's check rejects, unless the conf is true. The existing 4.4-to-5.0 note still covers the 5.0 refuse-to-start behavior, and the added lines contain no non-ASCII characters.
Findings
0 total: 0 P0, 0 P1, 0 P2, 0 P3.
No findings.
Generated by Omnigent on Databricks
|
Merge Summary:
Posted by |
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]>
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]> (cherry picked from commit 6a02fc4) Signed-off-by: Nicholas Chammas <[email protected]>
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]> (cherry picked from commit 6a02fc4) Signed-off-by: Nicholas Chammas <[email protected]>
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]> (cherry picked from commit 6a02fc4) Signed-off-by: Nicholas Chammas <[email protected]>
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]> (cherry picked from commit 6a02fc4) Signed-off-by: Nicholas Chammas <[email protected]>
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]> (cherry picked from commit 6a02fc4) Signed-off-by: Nicholas Chammas <[email protected]>
…st_target` trigger ### What changes were proposed in this pull request? Remove `notify_test_workflow.yml` and rely on the existing `update_build_status.yml` to report fork build status on PRs. Update the comments accordingly. This builds on [#58579](#58579) (SPARK-59293), which fixes errors that abort the scheduled scan. ### Why are the changes needed? ASF is disabling the `pull_request_target` trigger in [November 2026][asf], ahead of GitHub's [December 2 deadline][github]. This is our only workflow using it. The scheduled updater already creates missing `Build` checks (SPARK-58660), so we don't need another workflow to replace the notifier. Fork builds still run on push, as usual; this PR only changes how their status gets reported. We should backport this PR to active maintenance branches that accept PRs, since `pull_request_target` uses the base branch's workflow. [asf]: https://infra.apache.org/github-actions-policy.html [github]: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target I intentionally kept this PR simple as I _really_ don't like that we have to maintain so much GitHub Actions-specific JavaScript just to manage our fork-based contribution workflow. In other words, in the interest of simplicity I intentionally left out: - Any mechanism to update PRs "live". - Testing infrastructure for these workflows and the JavaScript they contain. This infrastructure is not in the Spark project's area of expertise. It really feels like something we should be able to import and use from somewhere. Until then, I think we should strive to keep these workflows as simple as possible. ### Does this PR introduce _any_ user-facing change? No, nothing for end users. For contributors here on GitHub: - The "Notify test workflow" check goes away. - New `Build` checks appear as `queued` on the first scheduled pass and pick up the fork result on the next. The schedule is every 15 minutes, but GitHub can delay or drop runs. - If you push multiple updates to a PR between scans, only the latest commit gets a `Build` check. - The results link opens the fork's Actions run instead of its `Run / Check changes` check. ### How was this patch tested? Checked locally with actionlint, YAML parsing, and git diff --check. `update_build_status.yml` changes only comments, so the scheduled updater, which runs from master, already behaves as it will after this PR. In [run 40517915808](https://github.com/apache/spark/actions/runs/40517915808) it created Build checks for PRs that had none (#54953, #59831), and later scheduled runs synced them (#54953 is now success). Additionally, an agent replayed the unmodified updater against 150 open PRs with their Build checks hidden and writes intercepted: all 150 resolve to the same fork run, and the 36 action_required PRs would get identical checks. All 708 open master PRs currently have exactly one Build check, and 706 match their fork run. This PR can't fully test itself because `pull_request_target` runs the base branch's copy of the notifier. I will run additional tests post merge. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Copilot SDK in VS Code (version unavailable) Closes #59829 from nchammas/pr-target-actions-policy. Authored-by: Nicholas Chammas <[email protected]> Signed-off-by: Nicholas Chammas <[email protected]> (cherry picked from commit 6a02fc4) Signed-off-by: Nicholas Chammas <[email protected]>
What changes were proposed in this pull request?
A follow-up to #58414, now that its
branch-4.xbackport #58438 has shippedspark.sql.hive.thriftServer.allowIneffectiveDoAsin 4.4.0 (as a warn-only gate):.version("5.0.0")to.version("4.4.0"), since 4.4.0 is the first release with this key.branch-4.x. That section is otherwise identical betweenmasterandbranch-4.x, and the 5.0 note added by [SPARK-59118][SQL] Error when doAs is used and not supported #58414 already says "Spark 4.4 logs a warning for the same configuration."Why are the changes needed?
A conf's version should record the first release that has it, and master's 4.3 to 4.4 migration section should match
branch-4.x's.Does this PR introduce any user-facing change?
No. Behavior is unchanged; the generated conf docs list the conf as available since 4.4.0 instead of 5.0.0.
How was this patch tested?
No new tests: the change is a version string and one documentation line. Checked that master's "Upgrading from Spark SQL 4.3 to 4.4" section is now identical to
branch-4.x's.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Isaac (Claude Opus 5.5)
This pull request and its description were written by Isaac.