Comparing changes

Signed-off-by: Dongjoon Hyun <[email protected]>

### What changes were proposed in this pull request? This PR aims to add `MacOS 15` to `macos-cpp-check` GitHub Action job additionally. ### Why are the changes needed? To have additional test coverage. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2218 from dongjoon-hyun/ORC-1895. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 1a2e3e0) Signed-off-by: Dongjoon Hyun <[email protected]>

…get evaluated correctly ### What changes were proposed in this pull request? When all values in column `col_0` are `NULL`s within a row group, and we attempt to apply the predicate pushdown `col_0 <=> 'xxx'`, the `evaluatePredicateProto` function returns `TruthValue.NULL`. In this case, we can directly determine the result based on the literal value: if the literal is `NULL`, return `TruthValue.YES`, otherwise, return `TruthValue.NO`. ### Why are the changes needed? See [SPARK-52032](https://issues.apache.org/jira/projects/SPARK/issues/SPARK-52032). When we pushdown the NULL_SAFE_EQUALS predicate, all values of the column are `NULL`. The `evaluatePredicateProto` returns `TruthValue.NULL`, whose `isNeeded` returns false so that the whole row group is skipped by `SargApplier.pickRowGroups`, which actually is incorrect. ### How was this patch tested? There already exists unit test -- `TestOrcTimezonePPD.testTimestampAllNulls` ### Was this patch authored or co-authored using generative AI tooling? Co-authored using generative AI tooling. Closes #2223 from jayhan94/fix_null_safe_equals_pred_push. Authored-by: Jay Han <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 467a293) Signed-off-by: Dongjoon Hyun <[email protected]>

### What changes were proposed in this pull request? This PR aims to upgrade Spark to 4.0.0 and Scala to 2.13.16. ### Why are the changes needed? To use the latest Apache Spark 4.0.0 instead of 4.0.0-preview2. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2230 from dongjoon-hyun/ORC-1899. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 1384bbd) Signed-off-by: Dongjoon Hyun <[email protected]>

### What changes were proposed in this pull request? This PR aims to upgrade `Jackson` to 2.18.2 in bench module. ### Why are the changes needed? To match with Apache Spark 4.0.0. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2233 from dongjoon-hyun/ORC-1900. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 2318ecb) Signed-off-by: Dongjoon Hyun <[email protected]>

…n` rule ### What changes were proposed in this pull request? This PR aims to remove `threeten-extra` exclusion in `enforceBytecodeVersion` rule. ### Why are the changes needed? The previous exclusion rule was added at ORC 1.9.0 with Java 1.8. It's not required for Java 17 since ORC 2.0. - #1536 ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2234 from dongjoon-hyun/ORC-1901. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 1d33d2a) Signed-off-by: Dongjoon Hyun <[email protected]>

Bumps [org.apache.maven.plugins:maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.5.3 to 3.6.0. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/maven-shade-plugin/commit/9a572e232d652a12a2404fe3cbe57fb588a87318"><code>9a572e2</code></a> [maven-release-plugin] prepare release maven-shade-plugin-3.6.0</li> <li><a href="https://github.com/apache/maven-shade-plugin/commit/ade2e359214c4094876b0100c97b9a55aecc4c2e"><code>ade2e35</code></a> [MSHADE-428] Prevent null value in array of transformers (<a href="https://redirect.github.com/apache/maven-shade-plugin/issues/229">#229</a>)</li> <li><a href="https://github.com/apache/maven-shade-plugin/commit/b573b8c498ea12cd2b84edb45be3036d9853d03e"><code>b573b8c</code></a> [MSHADE-478] Extra JARs feature (<a href="https://redirect.github.com/apache/maven-shade-plugin/issues/228">#228</a>)</li> <li><a href="https://github.com/apache/maven-shade-plugin/commit/199ffaecd26a912527173ed4edae366e48a00998"><code>199ffae</code></a> Drop the cruft (<a href="https://redirect.github.com/apache/maven-shade-plugin/issues/225">#225</a>)</li> <li><a href="https://github.com/apache/maven-shade-plugin/commit/912a81d95da870e2384359b2836caca4f2d1299f"><code>912a81d</code></a> Bump maven-gh-actions-shared to v4</li> <li><a href="https://github.com/apache/maven-shade-plugin/commit/80e44203f4ed2169b6186a29a7112aa8ed01c7a3"><code>80e4420</code></a> [maven-release-plugin] prepare for next development iteration</li> <li>See full diff in <a href="https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.5.3...maven-shade-plugin-3.6.0">compare view</a></li> </ul> </details> <br /> [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.apache.maven.plugins:maven-shade-plugin&package-manager=maven&previous-version=3.5.3&new-version=3.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `dependabot rebase` will rebase this PR - `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `dependabot merge` will merge this PR after your CI passes on it - `dependabot squash and merge` will squash and merge this PR after your CI passes on it - `dependabot cancel merge` will cancel a previously requested merge and block automerging - `dependabot reopen` will reopen this PR if it is closed - `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Closes #1953 from dependabot[bot]/dependabot/maven/java/org.apache.maven.plugins-maven-shade-plugin-3.6.0. Authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: Shaoyun Chen <[email protected]> (cherry picked from commit 1ba1ff6) Signed-off-by: Dongjoon Hyun <[email protected]>

### What changes were proposed in this pull request? This PR aims to remove unused test resource `log4j.properties` files. ### Why are the changes needed? Aapche ORC 1.7.5 switched the deprecated `slf4j-log4j12` to `slf4j-simple` via ORC-1182. So, these test resources are unused in ORC 1.7.6+. ``` $ find . -name log4j.properties ./java/bench/core/src/resources/log4j.properties ./java/tools/src/test/resources/log4j.properties ./java/core/src/test/resources/log4j.properties ./java/shims/src/test/resources/log4j.properties ./java/mapreduce/src/test/resources/log4j.properties ``` ### How was this patch tested? Pass the CIs and manual review because this doesn't affect ORC code. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2253 from dongjoon-hyun/ORC-1909. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 4431151) Signed-off-by: Dongjoon Hyun <[email protected]>

This PR aims to remove `Windows 2019` GitHub Action job. Note that ORC-1924 is created in order to recover Windows test coverage later. - #2273 `Windows 2019` is deprecated already and will be removed soon. - actions/runner-images#12045 Manual review because this is a removal of CI. No. Closes #2274 from dongjoon-hyun/ORC-1923. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 7e452b9) Signed-off-by: Dongjoon Hyun <[email protected]>

### What changes were proposed in this pull request? This PR aims to fix the javadoc of `ZstdCodec.compress` method. ### Why are the changes needed? The return type is `boolean` instead of `ZstdOptions`. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2285 from dongjoon-hyun/ORC-1929. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 7b5b1a7) Signed-off-by: Dongjoon Hyun <[email protected]>

…option ### What changes were proposed in this pull request? This PR aims to fix `PhysicalFsWriter` to change `tempOptions` directly. ### Why are the changes needed? In the following code path, `tempOptions` is supposed to be updated and used. However, `codec.getDefaultOptions() instanceof ZstdCodec.ZstdOptions options` code is currently updating the return value of `codec.getDefaultOptions()` via a variable `options`. https://github.com/apache/orc/blob/d3843bc043bea97bdd81a0f8e1fab3329efc7308/java/core/src/java/org/apache/orc/impl/PhysicalFsWriter.java#L118-L127 Technically, `ZstdCodec.getDefaultOptions()` returns the final static variable. This AS-IS code is trying to change this static object which leads unintended side-effects. We should avoid this code pattern. https://github.com/apache/orc/blob/d3843bc043bea97bdd81a0f8e1fab3329efc7308/java/core/src/java/org/apache/orc/impl/ZstdCodec.java#L150-L156 ### How was this patch tested? Pass the CIs with a newly added test case. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2304 from dongjoon-hyun/ORC-1942. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 3c63bf7) Signed-off-by: Dongjoon Hyun <[email protected]>

Signed-off-by: Dongjoon Hyun <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comparing changes

Open a pull request

Commits on May 4, 2025

Commits on May 7, 2025

Commits on May 21, 2025

Commits on May 23, 2025

Commits on Jun 9, 2025

Commits on Jun 17, 2025

Commits on Jun 18, 2025

Commits on Jun 30, 2025

Commits on Jul 4, 2025

This comparison is taking too long to generate.

Uh oh!