ORC-22. Fixes #3 - Allow InputStreams to set the natural read size for their filesystem #4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
their underlying file system. I've set the local file system to use 128k, but hdfs and webhdfs would both have substantially larger.
dongjoon-hyun
added a commit
that referenced
this pull request
Sep 16, 2021
### What changes were proposed in this pull request? This PR aims the following. - Use `Java 17` instead of `Java 17 EA`. - Remove `Java 16` to save the community testing resources - Recover GitHub Action CI by using `java17` profile. - Disable dependency check on `hadoop-hdfs` of `shim` module Note that there is a TODO (ORC-1003) which fails due to some `shaded` classes. We will revisit that. ### Why are the changes needed? Java 17 is finally available. ### How was this patch tested? Pass the CIs with Java 17. When we test manually, it passed like the following. ``` $ java -version openjdk version "17" 2021-09-14 LTS OpenJDK Runtime Environment Zulu17.28+13-CA (build 17+35-LTS) OpenJDK 64-Bit Server VM Zulu17.28+13-CA (build 17+35-LTS, mixed mode, sharing) ... Test project /Users/dongjoon/APACHE/orc-merge/build Start 1: orc-test 1/7 Test #1: orc-test ......................... Passed 3.41 sec Start 2: java-test 2/7 Test #2: java-test ........................ Passed 97.92 sec Start 3: java-tools-test 3/7 Test #3: java-tools-test .................. Passed 0.11 sec Start 4: java-bench-gen-test 4/7 Test #4: java-bench-gen-test .............. Passed 0.98 sec Start 5: java-bench-scan-test 5/7 Test #5: java-bench-scan-test ............. Passed 0.67 sec Start 6: java-bench-hive-test 6/7 Test #6: java-bench-hive-test ............. Passed 12.47 sec Start 7: tool-test 7/7 Test #7: tool-test ........................ Passed 10.02 sec 100% tests passed, 0 tests failed out of 7 Total Test time (real) = 125.58 sec Built target test-out ```
dongjoon-hyun
added a commit
that referenced
this pull request
Sep 16, 2021
### What changes were proposed in this pull request? This PR aims the following. - Use `Java 17` instead of `Java 17 EA`. - Remove `Java 16` to save the community testing resources - Recover GitHub Action CI by using `java17` profile. - Disable dependency check on `hadoop-hdfs` of `shim` module Note that there is a TODO (ORC-1003) which fails due to some `shaded` classes. We will revisit that. ### Why are the changes needed? Java 17 is finally available. ### How was this patch tested? Pass the CIs with Java 17. When we test manually, it passed like the following. ``` $ java -version openjdk version "17" 2021-09-14 LTS OpenJDK Runtime Environment Zulu17.28+13-CA (build 17+35-LTS) OpenJDK 64-Bit Server VM Zulu17.28+13-CA (build 17+35-LTS, mixed mode, sharing) ... Test project /Users/dongjoon/APACHE/orc-merge/build Start 1: orc-test 1/7 Test #1: orc-test ......................... Passed 3.41 sec Start 2: java-test 2/7 Test #2: java-test ........................ Passed 97.92 sec Start 3: java-tools-test 3/7 Test #3: java-tools-test .................. Passed 0.11 sec Start 4: java-bench-gen-test 4/7 Test #4: java-bench-gen-test .............. Passed 0.98 sec Start 5: java-bench-scan-test 5/7 Test #5: java-bench-scan-test ............. Passed 0.67 sec Start 6: java-bench-hive-test 6/7 Test #6: java-bench-hive-test ............. Passed 12.47 sec Start 7: tool-test 7/7 Test #7: tool-test ........................ Passed 10.02 sec 100% tests passed, 0 tests failed out of 7 Total Test time (real) = 125.58 sec Built target test-out ```
dongjoon-hyun
pushed a commit
that referenced
this pull request
Aug 17, 2022
### What changes were proposed in this pull request? This PR aims to add Febdora 37 Docker test. ### Why are the changes needed? To improve test coverage. ### How was this patch tested? Manually test on AMD64. Note that ORC-762 still exists; protobuffer 2.5.0 only works on AMD64 and Apple silicon. ``` $ docker build -t apache/orc-dev:fedora37 fedora37 $ ./run-one.sh local fedora37 fedora37 ... Test project /root/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 5.35 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 192.94 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.14 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 1.38 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 1.23 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 13.55 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 4.66 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 10.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 229.97 sec Built target test-out Finished fedora37 at Wed Aug 17 13:10:33 PDT 2022 ``` Closes #1224 from williamhyun/fedora37. Authored-by: William Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
pushed a commit
that referenced
this pull request
Aug 17, 2022
### What changes were proposed in this pull request? This PR aims to add Febdora 37 Docker test. ### Why are the changes needed? To improve test coverage. ### How was this patch tested? Manually test on AMD64. Note that ORC-762 still exists; protobuffer 2.5.0 only works on AMD64 and Apple silicon. ``` $ docker build -t apache/orc-dev:fedora37 fedora37 $ ./run-one.sh local fedora37 fedora37 ... Test project /root/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 5.35 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 192.94 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.14 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 1.38 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 1.23 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 13.55 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 4.66 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 10.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 229.97 sec Built target test-out Finished fedora37 at Wed Aug 17 13:10:33 PDT 2022 ``` Closes #1224 from williamhyun/fedora37. Authored-by: William Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 8f2b610) Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Aug 30, 2023
### What changes were proposed in this pull request? This PR aims to add `Debian 12` Docker test. ### Why are the changes needed? To improve the test coverage for the new Debian OS environment. ### How was this patch tested? Run the docker tests. ``` $ cd docker $ ./run-one.sh local ORC-1498 debian12 Started local run for ORC-1498 on debian12 at Tue Aug 29 23:50:19 PDT 2023 -- The C compiler identification is GNU 12.2.0 -- The CXX compiler identification is GNU 12.2.0 ... Java version: 17.0.8, vendor: Debian, runtime: /usr/lib/jvm/java-17-openjdk-arm64 ... Run CPack packaging tool... CPack: Create package using TGZ CPack: Install projects CPack: - Run preinstall target for: ORC CPack: - Install project: ORC [] CPack: Create package CPack: - package: /root/build/ORC-2.0.0-SNAPSHOT-Linux.tar.gz generated. Test project /root/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 4.61 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 105.33 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.08 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 0.73 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 0.65 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 11.11 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 3.23 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 6.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 132.45 sec Built target test-out Finished debian12 at Wed Aug 30 00:02:14 PDT 2023 ``` Closes #1605 from dongjoon-hyun/ORC-1498. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Aug 30, 2023
### What changes were proposed in this pull request? This PR aims to add `Debian 12` Docker test. ### Why are the changes needed? To improve the test coverage for the new Debian OS environment. ### How was this patch tested? Run the docker tests. ``` $ cd docker $ ./run-one.sh local ORC-1498 debian12 Started local run for ORC-1498 on debian12 at Tue Aug 29 23:50:19 PDT 2023 -- The C compiler identification is GNU 12.2.0 -- The CXX compiler identification is GNU 12.2.0 ... Java version: 17.0.8, vendor: Debian, runtime: /usr/lib/jvm/java-17-openjdk-arm64 ... Run CPack packaging tool... CPack: Create package using TGZ CPack: Install projects CPack: - Run preinstall target for: ORC CPack: - Install project: ORC [] CPack: Create package CPack: - package: /root/build/ORC-2.0.0-SNAPSHOT-Linux.tar.gz generated. Test project /root/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 4.61 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 105.33 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.08 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 0.73 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 0.65 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 11.11 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 3.23 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 6.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 132.45 sec Built target test-out Finished debian12 at Wed Aug 30 00:02:14 PDT 2023 ``` Closes #1605 from dongjoon-hyun/ORC-1498. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 3cd1837) Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Dec 9, 2023
### What changes were proposed in this pull request? This PR aims to add `Ubuntu 24.04 LTS` Docker Test. ### Why are the changes needed? Although the official `Ubuntu 24.04 LTS` will arrive on April 2024, we can start to track it from the official docker dev image. - GNU C/CXX compiler: 13.2.0 ### How was this patch tested? Manual review. ``` $ ./reinit.sh ubuntu24 $ ./run-one.sh local ORC-1541 ubuntu24 ... Test project /root/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 4.53 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 104.55 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.07 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 0.74 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 0.69 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 11.21 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 3.36 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 6.94 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 132.08 sec Built target test-out Finished ubuntu24 at Fri Dec 8 23:34:01 PST 2023 ``` This closes #1501 . Closes #1679 from dongjoon-hyun/ORC-1541. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Dec 24, 2023
### What changes were proposed in this pull request?
This PR aims to simplify `fedora37` docker image for Apache ORC 2.0.
- Explicitly install Java 17
- Remove unused Java8 and Java 11 installation
- Remove unused Maven installation.
### Why are the changes needed?
The docker image is outdated.
- Java 8 and 11-related branches are not used at all.
- The installed Java 17 was due to the transitive dependency of `maven` installation.
- The installed `Maven` is not used because we use `mvnw`.
### How was this patch tested?
Manual tests.
```
$ ./run-one.sh local main fedora37
...
Test project /root/build
Start 1: orc-test
1/8 Test #1: orc-test ......................... Passed 5.54 sec
Start 2: java-test
2/8 Test #2: java-test ........................ Passed 118.00 sec
Start 3: java-tools-test
3/8 Test #3: java-tools-test .................. Passed 0.09 sec
Start 4: java-bench-gen-test
4/8 Test #4: java-bench-gen-test .............. Passed 0.83 sec
Start 5: java-bench-scan-test
5/8 Test #5: java-bench-scan-test ............. Passed 0.74 sec
Start 6: java-bench-hive-test
6/8 Test #6: java-bench-hive-test ............. Passed 11.28 sec
Start 7: java-bench-spark-test
7/8 Test #7: java-bench-spark-test ............ Passed 3.49 sec
Start 8: tool-test
8/8 Test #8: tool-test ........................ Passed 7.92 sec
100% tests passed, 0 tests failed out of 8
Total Test time (real) = 147.89 sec
Built target test-out
Finished fedora37 at Sat Dec 23 19:32:19 PST 2023
```
Closes #1694 from dongjoon-hyun/ORC-1555.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Dec 24, 2023
### What changes were proposed in this pull request? This PR aims to add `Rocky Linux 9` Dockerfile. ### Why are the changes needed? To provide a test coverage for `Oracle 9`-compatible OS environment. - https://docs.rockylinux.org/release_notes/9_3/ - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/9.3_release_notes/index ### How was this patch tested? Manual test. ``` $ ./run-one.sh apache branch-1.9 rocky9 ... Test project /root/orc/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 4.92 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 114.57 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.07 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 0.75 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 0.71 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 11.45 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 3.26 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 6.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 142.44 sec Built target test-out Finished rocky9 at Sat Dec 23 23:21:18 PST 2023 ``` Closes #1695 from dongjoon-hyun/ORC-1556. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Dec 24, 2023
This PR aims to add `Rocky Linux 9` Dockerfile. To provide a test coverage for `Oracle 9`-compatible OS environment. - https://docs.rockylinux.org/release_notes/9_3/ - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/9.3_release_notes/index Manual test. ``` $ ./run-one.sh apache branch-1.9 rocky9 ... Test project /root/orc/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 4.92 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 114.57 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.07 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 0.75 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 0.71 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 11.45 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 3.26 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 6.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 142.44 sec Built target test-out Finished rocky9 at Sat Dec 23 23:21:18 PST 2023 ``` Closes #1695 from dongjoon-hyun/ORC-1556. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 9414da7) Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Dec 24, 2023
This PR aims to add `Rocky Linux 9` Dockerfile. To provide a test coverage for `Oracle 9`-compatible OS environment. - https://docs.rockylinux.org/release_notes/9_3/ - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/9.3_release_notes/index Manual test. ``` $ ./run-one.sh apache branch-1.9 rocky9 ... Test project /root/orc/build Start 1: orc-test 1/8 Test #1: orc-test ......................... Passed 4.92 sec Start 2: java-test 2/8 Test #2: java-test ........................ Passed 114.57 sec Start 3: java-tools-test 3/8 Test #3: java-tools-test .................. Passed 0.07 sec Start 4: java-bench-gen-test 4/8 Test #4: java-bench-gen-test .............. Passed 0.75 sec Start 5: java-bench-scan-test 5/8 Test #5: java-bench-scan-test ............. Passed 0.71 sec Start 6: java-bench-hive-test 6/8 Test #6: java-bench-hive-test ............. Passed 11.45 sec Start 7: java-bench-spark-test 7/8 Test #7: java-bench-spark-test ............ Passed 3.26 sec Start 8: tool-test 8/8 Test #8: tool-test ........................ Passed 6.70 sec 100% tests passed, 0 tests failed out of 8 Total Test time (real) = 142.44 sec Built target test-out Finished rocky9 at Sat Dec 23 23:21:18 PST 2023 ``` Closes #1695 from dongjoon-hyun/ORC-1556. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 9414da7) Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Sep 27, 2025
…fiers ### What changes were proposed in this pull request? This PR aims to fix `Timezone` to support legacy `US` TimeZone identifiers. ### Why are the changes needed? Since `Ubuntu 24.04` and `Debian 13` doesn't provide old `/usr/share/zoneinfo/US/*` files, ORC C++ library fails with the following error by default. It's misleading because both recent `IANA timezone database` and `TZDIR` cannot resolve this issue. We had better provide a workaround via aliases. > C++ exception with description "Time zone file /usr/share/zoneinfo/US/Pacific does not exist. > Please install IANA time zone database and set TZDIR env." thrown in the test body. Although there are many legacy timezone identifies, this PR aims to focus on `US` issues. For the rest of the code, we can handle it later based on the usage. - https://data.iana.org/time-zones/tzdb/backward ### How was this patch tested? Pass the CIs and manually run a docker test without these lines. https://github.com/apache/orc/blob/fbea8e016699ad8e7b318f5c793b4e48fe85af57/docker/ubuntu24/Dockerfile#L58 https://github.com/apache/orc/blob/fbea8e016699ad8e7b318f5c793b4e48fe85af57/docker/debian13/Dockerfile#L40 I verified locally with the revised `Debian 13` image. ``` $ docker run -it --rm apache/orc-dev:debian13 ls -al /usr/share/zoneinfo/US ls: cannot access '/usr/share/zoneinfo/US': No such file or directory $ ./run-one.sh local x debian13 Started local run for ORC-2011 on debian13 at Fri Sep 26 21:54:25 PDT 2025 -- The C compiler identification is GNU 14.2.0 -- The CXX compiler identification is GNU 14.2.0 ... Test project /root/build Start 1: orc-test 1/9 Test #1: orc-test ......................... Passed 7.24 sec Start 2: java-test 2/9 Test #2: java-test ........................ Passed 110.33 sec Start 3: java-examples-test 3/9 Test #3: java-examples-test ............... Passed 0.37 sec Start 4: java-tools-test 4/9 Test #4: java-tools-test .................. Passed 0.06 sec Start 5: java-bench-gen-test 5/9 Test #5: java-bench-gen-test .............. Passed 0.71 sec Start 6: java-bench-scan-test 6/9 Test #6: java-bench-scan-test ............. Passed 0.66 sec Start 7: java-bench-hive-test 7/9 Test #7: java-bench-hive-test ............. Passed 11.14 sec Start 8: java-bench-spark-test 8/9 Test #8: java-bench-spark-test ............ Passed 214.61 sec Start 9: tool-test 9/9 Test #9: tool-test ........................ Passed 5.00 sec 100% tests passed, 0 tests failed out of 9 Total Test time (real) = 350.16 sec Built target test-out Finished debian13 at Fri Sep 26 22:06:39 PDT 2025 ``` Please note that the test coverage should be added separately. In other words, the docker images should be updated **selectively and gradually** after this PR because the images are shared among multiple ORC branches. Since `Debian 13` is added newly for `main` and `branch-2.2` only, I'm planning to update the following after merging this PR to have a test coverage for this feature. https://github.com/apache/orc/blob/fbea8e016699ad8e7b318f5c793b4e48fe85af57/docker/debian13/Dockerfile#L40 ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2422 from dongjoon-hyun/ORC-2011. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Sep 27, 2025
…fiers ### What changes were proposed in this pull request? This PR aims to fix `Timezone` to support legacy `US` TimeZone identifiers. ### Why are the changes needed? Since `Ubuntu 24.04` and `Debian 13` doesn't provide old `/usr/share/zoneinfo/US/*` files, ORC C++ library fails with the following error by default. It's misleading because both recent `IANA timezone database` and `TZDIR` cannot resolve this issue. We had better provide a workaround via aliases. > C++ exception with description "Time zone file /usr/share/zoneinfo/US/Pacific does not exist. > Please install IANA time zone database and set TZDIR env." thrown in the test body. Although there are many legacy timezone identifies, this PR aims to focus on `US` issues. For the rest of the code, we can handle it later based on the usage. - https://data.iana.org/time-zones/tzdb/backward ### How was this patch tested? Pass the CIs and manually run a docker test without these lines. https://github.com/apache/orc/blob/fbea8e016699ad8e7b318f5c793b4e48fe85af57/docker/ubuntu24/Dockerfile#L58 https://github.com/apache/orc/blob/fbea8e016699ad8e7b318f5c793b4e48fe85af57/docker/debian13/Dockerfile#L40 I verified locally with the revised `Debian 13` image. ``` $ docker run -it --rm apache/orc-dev:debian13 ls -al /usr/share/zoneinfo/US ls: cannot access '/usr/share/zoneinfo/US': No such file or directory $ ./run-one.sh local x debian13 Started local run for ORC-2011 on debian13 at Fri Sep 26 21:54:25 PDT 2025 -- The C compiler identification is GNU 14.2.0 -- The CXX compiler identification is GNU 14.2.0 ... Test project /root/build Start 1: orc-test 1/9 Test #1: orc-test ......................... Passed 7.24 sec Start 2: java-test 2/9 Test #2: java-test ........................ Passed 110.33 sec Start 3: java-examples-test 3/9 Test #3: java-examples-test ............... Passed 0.37 sec Start 4: java-tools-test 4/9 Test #4: java-tools-test .................. Passed 0.06 sec Start 5: java-bench-gen-test 5/9 Test #5: java-bench-gen-test .............. Passed 0.71 sec Start 6: java-bench-scan-test 6/9 Test #6: java-bench-scan-test ............. Passed 0.66 sec Start 7: java-bench-hive-test 7/9 Test #7: java-bench-hive-test ............. Passed 11.14 sec Start 8: java-bench-spark-test 8/9 Test #8: java-bench-spark-test ............ Passed 214.61 sec Start 9: tool-test 9/9 Test #9: tool-test ........................ Passed 5.00 sec 100% tests passed, 0 tests failed out of 9 Total Test time (real) = 350.16 sec Built target test-out Finished debian13 at Fri Sep 26 22:06:39 PDT 2025 ``` Please note that the test coverage should be added separately. In other words, the docker images should be updated **selectively and gradually** after this PR because the images are shared among multiple ORC branches. Since `Debian 13` is added newly for `main` and `branch-2.2` only, I'm planning to update the following after merging this PR to have a test coverage for this feature. https://github.com/apache/orc/blob/fbea8e016699ad8e7b318f5c793b4e48fe85af57/docker/debian13/Dockerfile#L40 ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2422 from dongjoon-hyun/ORC-2011. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 3c89afe) Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Sep 27, 2025
### What changes were proposed in this pull request? This PR aims to remove `US` timezone workaround from `Debian 13` Docker image ### Why are the changes needed? We don't need this after the following fixes: - ORC-2010: Use `IANA` Identifier `America/Los_Angeles` instead of `US/Pacific` in Java - ORC-2011: [C++] Fix `Timezone` to support legacy `US` TimeZone identifiers ### How was this patch tested? Pass the CIs. Manually tests like the following. ``` $ cd docker $ ./reinit.sh debian13 $ ./run-one.sh local x debian13 Started local run for ORC-2012 on debian13 at Sat Sep 27 08:11:00 PDT 2025 -- The C compiler identification is GNU 14.2.0 -- The CXX compiler identification is GNU 14.2.0 ... Test project /root/build Start 1: orc-test 1/9 Test #1: orc-test ......................... Passed 7.05 sec Start 2: java-test 2/9 Test #2: java-test ........................ Passed 77.28 sec Start 3: java-examples-test 3/9 Test #3: java-examples-test ............... Passed 0.27 sec Start 4: java-tools-test 4/9 Test #4: java-tools-test .................. Passed 0.05 sec Start 5: java-bench-gen-test 5/9 Test #5: java-bench-gen-test .............. Passed 0.56 sec Start 6: java-bench-scan-test 6/9 Test #6: java-bench-scan-test ............. Passed 0.50 sec Start 7: java-bench-hive-test 7/9 Test #7: java-bench-hive-test ............. Passed 10.71 sec Start 8: java-bench-spark-test 8/9 Test #8: java-bench-spark-test ............ Passed 213.82 sec Start 9: tool-test 9/9 Test #9: tool-test ........................ Passed 4.67 sec 100% tests passed, 0 tests failed out of 9 Total Test time (real) = 314.92 sec Built target test-out Finished debian13 at Sat Sep 27 08:22:09 PDT 2025 ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2423 from dongjoon-hyun/ORC-2012. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Sep 27, 2025
### What changes were proposed in this pull request? This PR aims to remove `US` timezone workaround from `Debian 13` Docker image ### Why are the changes needed? We don't need this after the following fixes: - ORC-2010: Use `IANA` Identifier `America/Los_Angeles` instead of `US/Pacific` in Java - ORC-2011: [C++] Fix `Timezone` to support legacy `US` TimeZone identifiers ### How was this patch tested? Pass the CIs. Manually tests like the following. ``` $ cd docker $ ./reinit.sh debian13 $ ./run-one.sh local x debian13 Started local run for ORC-2012 on debian13 at Sat Sep 27 08:11:00 PDT 2025 -- The C compiler identification is GNU 14.2.0 -- The CXX compiler identification is GNU 14.2.0 ... Test project /root/build Start 1: orc-test 1/9 Test #1: orc-test ......................... Passed 7.05 sec Start 2: java-test 2/9 Test #2: java-test ........................ Passed 77.28 sec Start 3: java-examples-test 3/9 Test #3: java-examples-test ............... Passed 0.27 sec Start 4: java-tools-test 4/9 Test #4: java-tools-test .................. Passed 0.05 sec Start 5: java-bench-gen-test 5/9 Test #5: java-bench-gen-test .............. Passed 0.56 sec Start 6: java-bench-scan-test 6/9 Test #6: java-bench-scan-test ............. Passed 0.50 sec Start 7: java-bench-hive-test 7/9 Test #7: java-bench-hive-test ............. Passed 10.71 sec Start 8: java-bench-spark-test 8/9 Test #8: java-bench-spark-test ............ Passed 213.82 sec Start 9: tool-test 9/9 Test #9: tool-test ........................ Passed 4.67 sec 100% tests passed, 0 tests failed out of 9 Total Test time (real) = 314.92 sec Built target test-out Finished debian13 at Sat Sep 27 08:22:09 PDT 2025 ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2423 from dongjoon-hyun/ORC-2012. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit a6dfebd) Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun
added a commit
that referenced
this pull request
Oct 8, 2025
### What changes were proposed in this pull request? This PR aims to upgrade `CMake` to 3.26.0 in `amazonlinux:2023`. ### Why are the changes needed? Currently, it has 3.22.2 which is blocking #2416 . ``` $ docker run -it --rm apache/orc-dev:amazonlinux23 cmake --version cmake version 3.22.2 ``` ### How was this patch tested? Manual build and tests. ``` $ ./reinit.sh amazonlinux23 ``` ``` $ docker run -it --rm apache/orc-dev:amazonlinux23 cmake --version cmake version 3.26.0 CMake suite maintained and supported by Kitware (kitware.com/cmake). ``` ``` $ ./run-one.sh local x amazonlinux23 Started local run for main on amazonlinux23 at Wed Oct 8 00:03:37 PDT 2025 ... Test project /root/build Start 1: orc-test 1/9 Test #1: orc-test ......................... Passed 7.77 sec Start 2: java-test 2/9 Test #2: java-test ........................ Passed 118.90 sec Start 3: java-examples-test 3/9 Test #3: java-examples-test ............... Passed 0.43 sec Start 4: java-tools-test 4/9 Test #4: java-tools-test .................. Passed 0.07 sec Start 5: java-bench-gen-test 5/9 Test #5: java-bench-gen-test .............. Passed 0.77 sec Start 6: java-bench-scan-test 6/9 Test #6: java-bench-scan-test ............. Passed 0.71 sec Start 7: java-bench-hive-test 7/9 Test #7: java-bench-hive-test ............. Passed 11.41 sec Start 8: java-bench-spark-test 8/9 Test #8: java-bench-spark-test ............ Passed 216.61 sec Start 9: tool-test 9/9 Test #9: tool-test ........................ Passed 4.37 sec 100% tests passed, 0 tests failed out of 9 Total Test time (real) = 361.08 sec Built target test-out Finished amazonlinux23 at Wed Oct 8 00:16:38 PDT 2025 ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #2435 from dongjoon-hyun/ORC-2016. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've set the local file system to use
128k, but hdfs and webhdfs would both have substantially larger.