Skip to content

docs: fix SQL examples that the default dialect cannot parse - #26373

Merged
neilconway merged 2 commits into
apache:mainfrom
rawsun007:docs-sql-examples-parse
Oct 23, 2026
Merged

neilconway merged 2 commits into
apache:mainfrom
rawsun007:docs-sql-examples-parse

Conversation

@rawsun007

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

  • None; documentation fix.

  • Closes #.

Rationale for this change

Two SQL examples in the user guide fail to parse with the default dialect:

  • ddl.md, "Example: Compression": COMPRESSION TYPE GZIP now raises ParserError("COMPRESSION TYPE clause is no longer in use. Please use the OPTIONS clause with 'format.compression' set appropriately ...").
  • struct_coercion.md writes struct types as STRUCT(a INT, b INT), in both CREATE TABLE column definitions and CAST. Only the DuckDB dialect parses that form; the default dialect gives ParserError("Expected: ',' or ')' after column definition, found: ("). Nine of the page's SQL blocks fail this way.

What changes are included in this PR?

  • The compression example sets 'format.compression' 'gzip' in OPTIONS instead.
  • The struct coercion page uses STRUCT<a INT, b INT>, which the default dialect accepts. The struct() value constructor is unchanged.

What is the testing strategy for this PR?

Ran every SQL block on both pages with datafusion-cli built from main, default dialect. Before, the compression example and nine struct blocks failed to parse; after, the remaining errors are fragments that reference tables or columns from other blocks. The compression example reads a gzipped CSV back correctly. ./ci/scripts/doc_prettier_check.sh passes. Docs only, no test.

Are there any user-facing changes?

Documentation only.

Generated with Claude Code (claude-opus-5-5). It came from parsing the SQL examples in the docs against main, not from an issue.

The COMPRESSION TYPE clause now raises a parser error, so the
compressed-CSV example uses 'format.compression' in OPTIONS. The struct
coercion page wrote struct types as STRUCT(a INT), which only the DuckDB
dialect accepts; use STRUCT<a INT>.
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 15, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.85%. Comparing base (c52eb97) to head (567fa38).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26373      +/-   ##
==========================================
- Coverage   82.85%   82.85%   -0.01%     
==========================================
  Files        1151     1151              
  Lines      455996   455996              
  Branches   455996   455996              
==========================================
- Hits       377805   377797       -8     
- Misses      55239    55244       +5     
- Partials    22952    22955       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@AdamGS AdamGS left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch!

@AdamGS
AdamGS added this pull request to the merge queue Oct 22, 2026
@alamb alamb added the Prolific Author author opened PRs or issues in more than 20 repositories in the last 30 days label Oct 22, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 22, 2026
@neilconway
neilconway enabled auto-merge October 23, 2026 01:27
@neilconway
neilconway added this pull request to the merge queue Oct 23, 2026
Merged via the queue into apache:main with commit 30f066d Oct 23, 2026
46 checks passed
zhuqi-lucas pushed a commit to zhuqi-lucas/arrow-datafusion that referenced this pull request Oct 28, 2026
## Which issue does this PR close?

Related to apache#25148. Inspired by apache/airflow#75619.

## Rationale for this change

Merge queue runs continue using runners after their group is removed or
replaced. The Rust workflow gives each merge-group SHA a separate
concurrency group, so a replacement does not cancel the old run. Jobs
still waiting for a runner can start long after the queue has stopped
using their results.

A read-only analysis of runs created on October 19–25, 2026 (UTC) found:

| Measurement | Result |
| --- | ---: |
| Merge-group workflow runs inspected | 1,170 |
| Distinct merge-group commits / PRs | 195 / 154 |
| Removed or superseded groups with subsequent runner use | 36 |
| Workflows / jobs with subsequent runner use | 91 / 473 |
| Runner time after removal or replacement | 20.3 hours |
| Jobs that started after removal or replacement | 312, using 12.7
runner-hours |

For the Rust and extended-test workflows, this was 19.0 of 369.4
runner-hours, or 5.2% of their sampled runner time. Cancellation taking
effect one minute after obsolescence would have avoided 17.9
runner-hours; at five minutes, 11.9 hours; at a fixed fifteen-minute
delay, 9.4 hours. A separate model using the next exact quarter-hour
boundary estimates 11.2 hours.

Examples:

- [apache#26562](apache#26562): after
manual queue removal, 34 jobs used another 121.6 runner-minutes in
parallel. They finished within about 14 minutes. [Rust
run](https://github.com/apache/datafusion/actions/runs/40069843292).
- [apache#26373](apache#26373): the group
was removed for failed checks at 21:21 UTC on October 22, but its MSRV
job did not start until 23:20 UTC. [Rust
run](https://github.com/apache/datafusion/actions/runs/39855621556).

The cutoff was an exact-SHA queue-removal event or the creation of a
replacement group for the same PR. For each job, the analysis summed
time after the later of its start and the cutoff plus the assumed
cancellation delay. It excluded skipped jobs, deduplicated successful
jobs copied into rerun records, and excluded seven rerun attempts
started after obsolescence that could have been deliberate debugging.
All Rust and extended-test jobs were fetched for the denominator; other
workflows were inspected when their run metadata showed activity after
the cutoff. Merged groups and all rerun attempts contributed zero to the
reported avoidable time, so preserving both leaves these estimates
unchanged.

These are historical runner times and estimated avoidable portions, not
deployed savings or billing estimates. Of the 312 late-starting jobs, 42
used Runs-On (2.4 runner-hours); 270 used GitHub Linux runners. We have
not measured whether cancelling queued jobs immediately terminates
instances already starting. The models also exclude ref-deletion
latency, cancellation cleanup, scheduler delays, and cleanup-workflow
overhead. Recent runner changes make this sample unsuitable as an annual
forecast.

## What changes are included in this PR?

- Scan when a new merge group is created and every 15 minutes. Ref
deletion can lag the event, so prompt cancellation is not guaranteed.
- Ship in observation mode. Cancellation requires the repository Actions
variable `MERGE_QUEUE_CLEANUP_ENABLED=true`. Document at least one week
of observation before enabling it; there is no automatic activation.
- Log candidate run IDs, SHAs, attempts, reasons, and UTC observation
times for comparison with eventual outcomes. Each merge-group scan has
its own concurrency group, keyed by the group SHA, so a waiting scan on
a group commit is never replaced and shown as cancelled. Scheduled and
manual scans keep one group per mode, so a manual dry run cannot replace
a waiting cancellation scan.
- Target only first-attempt, unfinished `merge_group` runs in
`apache/datafusion` whose exact queue ref is missing or points
elsewhere. Recheck the run and ref, then compare with its target branch
to preserve commits already merged. Preserve reruns, other event types,
and cleanup itself.
- Keep exact-ref detection while the GraphQL queue-entry-to-group-SHA
mapping remains unvalidated. A changed ref target is a safeguard for a
reused queue branch; the suffix is not treated as the run SHA.
- If an Actions run listing reaches the 1,000-result cap, stop the scan
without cancelling any runs. Active merge-group runs stay far below the
cap.
- Share the ref lookup and the target-branch comparison among the
workflows of one group.
- API errors preserve affected runs and allow other candidates to be
checked. Merge-group scans warn; scheduled/manual scans fail visibly.
Recheck HTTP 409 responses and retry unresolved cancellations on later
scans.
- Check out the default branch's script. The workflow definition itself
still comes from the triggering revision.
- Add behavioral tests to Dev CI and the local lint suite; remove the
test that repeated YAML literals.

## What is the testing strategy for this PR?

- 35 cleanup tests cover active/merged groups, missing/replaced refs,
target branches containing slashes, reruns, races, dry runs, API errors,
error reporting by trigger, pagination, the 1,000-result cap, and shared
comparisons. Together with existing fork-CI tests, 59 Node tests pass.
- Mutation checks confirm the merged-commit, rerun, listing-cap and
shared-comparison regressions are detected.
- `cargo fmt --all`, `cargo clippy --all-targets --all-features -- -D
warnings`, and the complete `./dev/rust_lint.sh` suite pass, including
generated docs, links, dependency checks, Rust docs, and the HTML
documentation build.
- actionlint 1.7.12 passes for both changed workflows.
- A live read-only probe completed five listing calls with no eligible
candidates. No workflow was cancelled during validation. The observation
period, production cancellation latency, and instance shutdown behavior
remain to be validated after deployment.

## Are there any user-facing changes?

Initially the workflow reports candidates only. Once enabled by a
maintainer, it cancels obsolete first-attempt merge-queue runs while
preserving merged commits and deliberate reruns. Contributors can rerun
obsolete jobs for debugging. No query behavior or public API changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation Prolific Author author opened PRs or issues in more than 20 repositories in the last 30 days v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants