@@ -97,34 +97,142 @@ A PR that quietly marks something `Compatible` while the diff shows a known dive
9797most important thing to catch. Keep reasons concise and link a tracking issue when the behavior is
9898known to differ.
9999
100+ Two disguised forms of the same thing:
101+
102+ - A divergence documented in the compatibility guide on a path that is enabled by default. If the
103+ guide says the results differ, the default has to fall back for that case (#5469 , #5507 ).
104+ - A test switched to ` ignore ` in the PR because Comet's answer now differs from Spark's. An ignored
105+ test is a known divergence, so the code needs a fallback for that case (#5701 ).
106+
107+ ### Behavior change against the latest release
108+
109+ Users upgrade from a release, not from ` main ` . So the question is not only what the PR changes
110+ relative to ` main ` , but whether a user moving from the latest release to a build with this PR sees
111+ different behavior. ` main ` may already carry unreleased changes in the same code, and a PR that
112+ looks harmless against ` main ` can finish turning a released behavior into a different one.
113+
114+ Find the latest release branch, the highest ` branch-X.Y ` :
115+
116+ ``` shell
117+ git ls-remote --heads https://github.com/apache/datafusion-comet ' branch-*' | sort -t- -k2 -V | tail -1
118+ ```
119+
120+ Fetch the PR head and diff the files the PR touches against that branch, not against ` main ` :
121+
122+ ``` shell
123+ git fetch https://github.com/apache/datafusion-comet branch-X.Y:release-X.Y pull/< pr> /head:pr-< pr>
124+ git diff release-X.Y pr-< pr> -- < changed files>
125+ ```
126+
127+ Then ask, for each code path the PR touches, whether any of these differ from the release:
128+
129+ - the result for some input, including null, NaN, ` -0.0 ` , empty, overflow, and timezone cases
130+ - whether a query raises an error, and which error
131+ - whether an operator or expression runs natively or falls back to Spark
132+ - a config default, a config name, or a support level
133+ - performance or memory use on an existing path
134+
135+ The tests the PR adds are a good probe. If a new test would fail on the release branch, the PR
136+ changes released behavior, and you should know which of those two cases it is. Running the test
137+ against the release branch is the cheapest way to find out when the answer is not obvious from
138+ the code.
139+
140+ Every behavior change against the release is one of two things:
141+
142+ - ** Intended.** A bug fix that makes Comet match Spark, or a deliberate change. The PR description
143+ should say so, user-facing changes need a note in the user guide or compatibility docs, and a
144+ correctness fix should be considered for backport to the release branches per
145+ ` docs/source/contributor-guide/backporting.md ` .
146+ - ** Unintended.** The PR, alone or together with unreleased changes already on ` main ` , makes Comet
147+ diverge from Spark where the release did not, or makes a released path slower. This is a
148+ correctness or performance regression, and it falls under the request-changes rule below.
149+
150+ Look hardest when the PR is one of the three kinds of change behind most of the regressions that the
151+ 1.1.0 audit found ([ #6399 ] ( https://github.com/apache/datafusion-comet/issues/6399 ) ):
152+
153+ - ** A path that becomes native by default.** The query used to fall back to Spark and was right.
154+ Find the inputs where the native path differs from Spark, and either test them or fall back for
155+ them. Making map and struct literals native exposed a type mismatch in the native ` IF ` (#6334 ).
156+ Constant metadata columns went native with a per-split value that DataFusion and Spark assign
157+ differently (#6505 ).
158+ - ** A removed fallback or guard.** List everything the fallback was shielding, not just the case the
159+ PR is about. Removing the Iceberg complex-type null-check fallback also exposed every ` explode ` of
160+ an Iceberg array to a schema-evolution bug in iceberg-rust (#6504 ).
161+ - ** A broad routing change.** A catch-all that sends more expressions down a path, such as the JVM
162+ codegen dispatcher, admits shapes the path never handled. List the new shapes and test one of each
163+ (#6424 , #6425 ).
164+
165+ Report the comparison in the review even when nothing changed, so the reviewer knows it was done.
166+
100167### Spark version coverage
101168
102169Comet supports several Spark versions. Version-specific behavior belongs in the shims under
103170` spark/src/main/spark-{3.4,3.5,3.x,4.0,4.1,4.2}/org/apache/comet/shims/ ` , not in branches on a
104171version string in shared code, and not in native Rust. If the PR adds a shim for one 4.x version,
105172check that the sibling 4.x source sets got it too.
106173
174+ When Spark changed the behavior in a patch release, such as SPARK-55969 or SPARK-54918, a check on
175+ the minor version is wrong for every earlier patch. CI builds only the newest patch of each line, so
176+ it can't catch that (#6042 , #5701 ). The pull request CI also runs only the default Spark profile, so
177+ logic that depends on the Spark version needs the matching ` run-spark-* ` labels (#6156 ).
178+
107179### Configuration
108180
109181New configs go in ` CometConf.scala ` and must follow
110182` docs/source/contributor-guide/config_conventions.md ` . Check the naming, the default, the category,
111183and that the description reads as user-facing documentation, because it is. ` configs.md ` is
112184generated from it.
113185
186+ A new behavior that is on by default and trades performance for some workloads needs a supported
187+ config, not a testing one, to turn it off before merge (#6466 ).
188+
114189### Tests
115190
116191- Does the PR test the thing it changed, or only that nothing else broke?
117192- Are the tests in the right framework for the area? The area skills say which.
118193- Does a bug fix come with a test that fails without the fix?
194+ - Do the tests compare against Spark? A comparison with Comet's own accumulator, or with a second
195+ implementation such as iceberg-rust instead of the Iceberg Java that Spark runs, can agree on the
196+ wrong answer (#6423 , #6426 ).
119197- New suites must be registered in both ` .github/workflows/pr_build_linux.yml ` and
120198 ` pr_build_macos.yml ` .
121199
200+ For a change on a default path, look for tests with the inputs that broke earlier changes:
201+
202+ - ` -0.0 ` next to ` 0.0 ` , and NaN with different payloads, including inside arrays and structs (#5469 ,
203+ #5507 , #5701 )
204+ - Values at a type or unit boundary, including negative timestamps before 1970 (#6426 )
205+ - A constant that binary floating point can't represent, such as 0.1, aggregated across partitions
206+ (#6423 )
207+ - A batch where every row takes the same branch (#6334 )
208+ - Output that the native side slices, so it reaches the JVM past the first batch with a non-zero
209+ offset (#6464 )
210+ - A Parquet file that Spark splits into several partitions, an empty file, and an Iceberg file
211+ written before a nested field was added (#6504 , #6505 , #6506 )
212+ - Data that defeats a heuristic, such as distinct keys at the start of a task followed by repeats
213+ (#6466 )
214+ - One of a pair of aliases configured without the other (#5825 )
215+
122216### CI
123217
124218` gh pr checks <pr> --repo apache/datafusion-comet ` , and again with ` --failed ` for detail.
125219
126220Summarize failures in the review. Do not compare against failures on ` main ` .
127221
222+ Some paths have no CI coverage at all. Nothing in CI reads from a real object store, runs Iceberg's
223+ forward-compatibility tables, or makes a spilled operator replay under a tight memory pool. A change
224+ on one of those paths needs a local run, and the review should ask for one (#5759 , #6254 ).
225+
226+ ### Dependency upgrades
227+
228+ Review a dependency bump as a set of behavior changes, not an API migration. For DataFusion, arrow,
229+ parquet and iceberg-rust, diff the upstream source of what Comet calls on default paths. Both
230+ versions are in the local Cargo registry after a build of each side. A test that the bump changes to
231+ ` ignore ` or to a new expected value marks a behavior change that needs a fallback or an explicit
232+ decision. The DataFusion 55 upgrade changed signed-zero handling in ` array_distinct ` and
233+ ` array_union ` (#5701 ), the spill replay of the final aggregate (#6254 ), and, through iceberg-rust,
234+ task validation (#5759 ).
235+
128236## 6. Documentation Freshness
129237
130238Every PR carries a documentation question, and it has two halves.
@@ -192,13 +300,16 @@ Present your review as guidance for the reviewer:
1923001 . ** PR Summary** , brief description of what the PR does
1933012 . ** Areas** , which sibling skills you loaded and why
1943023 . ** CI Status** , summary of CI check results
195- 4 . ** Findings** , organized by area
196- 5 . ** Suggested Review Comments** , specific comments the reviewer could leave, with file and line
303+ 4 . ** Behavior vs Release** , which release branch you compared against, and every behavior change
304+ you found, each marked intended or unintended. Say "no change" when there is none.
305+ 5 . ** Findings** , organized by area
306+ 6 . ** Suggested Review Comments** , specific comments the reviewer could leave, with file and line
197307 references. Everything here is something you expect the author to address. Anything that did not
198308 clear the bar above should not appear.
199- 6 . ** Review State** , how to submit the review. Use the first case that applies:
309+ 7 . ** Review State** , how to submit the review. Use the first case that applies:
200310 - ** Request changes** when any finding is a correctness problem that the PR introduces or a
201- performance regression. Name those findings.
311+ performance regression, including an unintended behavior change against the latest release.
312+ Name those findings.
202313 - ** Approve** , or dismiss the earlier review, when this is a re-review and the findings behind the
203314 reviewer's earlier ** Request changes** review have all been addressed.
204315 - ** Comment** otherwise.
0 commit comments