package yum: keep SRPM paths for backward compatibility#2558
package yum: keep SRPM paths for backward compatibility#2558kou merged 15 commits intogroonga:mainfrom
Conversation
82c5328 to
7c36b73
Compare
Apache Arrow changed SRPM output location from 'Source/Packages/' to 'source/SRPMS/'. This change breaks Yum repository compatibility for existing AlmaLinux distributions. This commit adds path management after yum:build to maintain compatibility: - For AlmaLinux 10 and earlier: Rename source/SRPMS/ to Source/Packages/ - For AlmaLinux 11 and later: Uses the new source/SRPMS/ structure - For aarch64 architecture: Removes SRPMs
7c36b73 to
5e0869f
Compare
Before this PR$ tree packages-almalinux-8-x86_64.new/
packages-almalinux-8-x86_64.new/
├── almalinux-8-x86_64
│ └── packages
│ └── yum
│ └── repositories
│ └── almalinux
│ └── 8
│ ├── Source
│ │ └── Packages
│ │ └── groonga-15.1.8-1.el8.src.rpm
│ └── x86_64
│ └── Packages
│ ├── groonga-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-debugsource-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-devel-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-doc-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-examples-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-libs-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-libs-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-munin-plugins-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-h3-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-suggest-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-server-common-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-server-gqtp-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-server-http-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-token-filter-stem-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-tokenizer-mecab-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.x86_64.rpm
│ └── groonga-tools-15.1.8-1.el8.x86_64.rpm
└── almalinux-8-x86_64.tar.gz
11 directories, 23 filesAfter this PR$ tree packages-almalinux-8-x86_64/
packages-almalinux-8-x86_64/
├── almalinux-8-x86_64
│ └── packages
│ └── yum
│ └── repositories
│ └── almalinux
│ └── 8
│ ├── source
│ │ └── SRPMS
│ │ └── groonga-15.1.8-1.el8.src.rpm
│ └── x86_64
│ └── Packages
│ ├── groonga-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-debugsource-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-devel-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-doc-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-examples-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-libs-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-libs-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-munin-plugins-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-h3-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-suggest-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-server-common-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-server-gqtp-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-server-http-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-token-filter-stem-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-tokenizer-mecab-15.1.8-1.el8.x86_64.rpm
│ ├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.x86_64.rpm
│ └── groonga-tools-15.1.8-1.el8.x86_64.rpm
└── almalinux-8-x86_64.tar.gz
11 directories, 23 files |
|
|
||
| def yum_build(console: false) | ||
| super | ||
| manage_srpm_paths_for_compatibility |
There was a problem hiding this comment.
manageってどういうニュアンスで使っていますか?
私は「なんとかする」くらいの意味だと思っています。もしそういうニュアンスで使っているなら、そういうふわっとした意味よりもっと明確な操作を示す単語を使って欲しいです。
keep_srpm_path_compatibilityとか?
There was a problem hiding this comment.
fix: e015dc0 2回同じループを回すのが引っかかったので、manage_srpm_pathsで互換性の処理とARM64環境の対応を合わせて対応させてみました。
(変更前のほうが何をするかすぐにわかりやすいかなとも思っているので少し悩んでいます。)
There was a problem hiding this comment.
遅い処理でもなく、速度が求められる処理でもないので、読みやすさを犠牲にするメリットがないです。そのため、一緒にしなくていいっす。
こういう読みやすさを犠牲にする最適化をするべきかどうかという判断をするときは、そもそも遅くて困っているのか?困っているならどのくらい速くしないといけないのかとかを考えるべきです。一般的には。
たとえば、こういう開発用のツールとか、そもそもループで回すものが少ないときは、最適化する必要がないことが多いです。一方、Groongaの全文検索関連の処理とかは速さがマジで重要で、扱うデータ量も多いので、読みやすさを犠牲にしてでもがんばらないといけないという判断が必要なときが多々あります。
There was a problem hiding this comment.
fix: 22db6ba
ありがとうございます。今後、Groongaだけに閉じずに開発する際に意識していこうと思います!
| # Backward compatibility: only rename for existing distributions that | ||
| # still expect the legacy SRPM structure. | ||
| # Legacy (<= AlmaLinux 10): source/SRPMS | ||
| # New (>= AlmaLinux 11): Source/Packages (no rename needed) | ||
| next if distribution != "almalinux" |
There was a problem hiding this comment.
この条件だとAlmaLinux 11でもリネームしませんか?
現状で対応しているディストリビューションが以下なので、これらはリネームし、それ以外は何もしない、がいいです。
Lines 67 to 78 in cb83ebf
今の実装だとAmazonLinuxのことは考慮していないですよね。
| # Backward compatibility: only rename for existing distributions that | |
| # still expect the legacy SRPM structure. | |
| # Legacy (<= AlmaLinux 10): source/SRPMS | |
| # New (>= AlmaLinux 11): Source/Packages (no rename needed) | |
| next if distribution != "almalinux" | |
| # Rename SRPM directory only for the following distributions | |
| # to keep backward compatibility: | |
| # | |
| # * AlmaLinux 8 | |
| # * AlmaLinux 9 | |
| # * AlmaLinux 10 | |
| # * Amazon Linux 2023 | |
| # | |
| # They keep using source/SRPMS/*.src.rpm for SRPM path. | |
| case "#{distribution}-#{version}" | |
| when "almalinux-8", | |
| "almalinux-9", | |
| "almalinux-10", | |
| "amazon-linux-2023" | |
| next | |
| end |
There was a problem hiding this comment.
fix: af2933d 対応するディストリビューションを明示的に記載するように修正しています。
| repo_base = "#{repositories_dir}/#{distribution}/#{version}" | ||
| old_path = "#{repo_base}/source/SRPMS" | ||
| new_path = "#{repo_base}/Source/Packages" |
There was a problem hiding this comment.
old_path/new_pathにするならrepo_baseもXXX_pathにしませんか?
| repo_base = "#{repositories_dir}/#{distribution}/#{version}" | |
| old_path = "#{repo_base}/source/SRPMS" | |
| new_path = "#{repo_base}/Source/Packages" | |
| base_path = "#{repositories_dir}/#{distribution}/#{version}" | |
| old_path = "#{base_path}/source/SRPMS" | |
| new_path = "#{base_path}/Source/Packages" |
There was a problem hiding this comment.
fix: 15485bb pathの名前に関しては一貫して、xx_pathを利用するように修正しました。
|
ここまで頂いたレビューコメントの対応を行いました。 動作確認ログ$ tree almalinux-8-x86_64/
almalinux-8-x86_64/
└── packages
└── yum
└── repositories
└── almalinux
└── 8
├── source
│ └── SRPMS
│ └── groonga-15.1.8-1.el8.src.rpm
└── x86_64
└── Packages
├── groonga-15.1.8-1.el8.x86_64.rpm
├── groonga-debuginfo-15.1.8-1.el8.x86_64.rpm
├── groonga-debugsource-15.1.8-1.el8.x86_64.rpm
├── groonga-devel-15.1.8-1.el8.x86_64.rpm
├── groonga-doc-15.1.8-1.el8.x86_64.rpm
├── groonga-examples-15.1.8-1.el8.x86_64.rpm
├── groonga-libs-15.1.8-1.el8.x86_64.rpm
├── groonga-libs-debuginfo-15.1.8-1.el8.x86_64.rpm
├── groonga-munin-plugins-15.1.8-1.el8.x86_64.rpm
├── groonga-plugin-h3-15.1.8-1.el8.x86_64.rpm
├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.x86_64.rpm
├── groonga-plugin-suggest-15.1.8-1.el8.x86_64.rpm
├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.x86_64.rpm
├── groonga-server-common-15.1.8-1.el8.x86_64.rpm
├── groonga-server-gqtp-15.1.8-1.el8.x86_64.rpm
├── groonga-server-http-15.1.8-1.el8.x86_64.rpm
├── groonga-token-filter-stem-15.1.8-1.el8.x86_64.rpm
├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.x86_64.rpm
├── groonga-tokenizer-mecab-15.1.8-1.el8.x86_64.rpm
├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.x86_64.rpm
└── groonga-tools-15.1.8-1.el8.x86_64.rpm
10 directories, 22 files$ tree almalinux-8-aarch64/
almalinux-8-aarch64/
└── packages
└── yum
└── repositories
└── almalinux
└── 8
└── aarch64
└── Packages
├── groonga-15.1.8-1.el8.aarch64.rpm
├── groonga-debuginfo-15.1.8-1.el8.aarch64.rpm
├── groonga-debugsource-15.1.8-1.el8.aarch64.rpm
├── groonga-devel-15.1.8-1.el8.aarch64.rpm
├── groonga-doc-15.1.8-1.el8.aarch64.rpm
├── groonga-examples-15.1.8-1.el8.aarch64.rpm
├── groonga-libs-15.1.8-1.el8.aarch64.rpm
├── groonga-libs-debuginfo-15.1.8-1.el8.aarch64.rpm
├── groonga-munin-plugins-15.1.8-1.el8.aarch64.rpm
├── groonga-plugin-h3-15.1.8-1.el8.aarch64.rpm
├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.aarch64.rpm
├── groonga-plugin-suggest-15.1.8-1.el8.aarch64.rpm
├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.aarch64.rpm
├── groonga-server-common-15.1.8-1.el8.aarch64.rpm
├── groonga-server-gqtp-15.1.8-1.el8.aarch64.rpm
├── groonga-server-http-15.1.8-1.el8.aarch64.rpm
├── groonga-token-filter-stem-15.1.8-1.el8.aarch64.rpm
├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.aarch64.rpm
├── groonga-tokenizer-mecab-15.1.8-1.el8.aarch64.rpm
├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.aarch64.rpm
└── groonga-tools-15.1.8-1.el8.aarch64.rpm
8 directories, 21 files |
|
fix: ce5b45b I will fix typos. |
f71e64c to
4625aee
Compare
4625aee to
e015dc0
Compare
This reverts commit e015dc0. We don't have to persue the speed here because there are not a lot of distibutions. Readability is more important.
| else | ||
| next |
There was a problem hiding this comment.
明示的にコメントだけ入れるようにしてみました。
" # Use latest SRPM path for other distributions Source/Packages/*.src.rpm"
| repositories_path = "#{yum_dir}/repositories" | ||
| yum_targets.each do |target| | ||
| distribution, version, architecture = split_target(target) | ||
| next if architecture != "aarch64" |
There was a problem hiding this comment.
やりたいことはSRPMは1つだけ残して全部消したい、なので、aarch64だけ消す、じゃなくて、x86_64以外は消すというコードにしませんか?(オリジナルのコードも[ "${ARCHITECTURE}" != "x86_64" ]なのでそういう意図のコードになっていました。)
x86_64のときはarchitectureがnilになる気がするのでnext unless architecture.nil?かな。
There was a problem hiding this comment.
やりたいことはSRPMは1つだけ残して全部消したい
fix: 2c13cb2 ありがとうございます。なるほどです。aarch64の場合だけ削除すると捉えてしまっていたのですが、本質的には何をやりたいのかちゃんと理解できていなかったですmm
|
ここまで頂いたレビューコメントの対応を行いました。 |
There was a problem hiding this comment.
Pull Request Overview
This PR refactors SRPM (Source RPM) path management from a shell script approach to Ruby methods to handle backward compatibility with different AlmaLinux distributions following Apache Arrow's SRPM location changes.
- Moves SRPM path management logic from GitHub Actions workflow to Ruby methods in the package task
- Implements distribution-specific path handling for AlmaLinux 8-10 and Amazon Linux 2023
- Replaces shell-based SRPM removal for non-x86_64 architectures with Ruby implementation
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| packages/packages-groonga-org-package-task.rb | Adds Ruby methods for SRPM path compatibility and architecture-specific removal |
| .github/workflows/package.yml | Removes shell script logic for SRPM handling from the workflow |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| old_path = "#{distribution_path}/source/SRPMS" | ||
| new_path = "#{distribution_path}/Source/Packages" | ||
| mkdir_p(File.dirname(old_path)) | ||
| mv(new_path, old_path) | ||
| rm_rf(File.dirname(new_path)) |
There was a problem hiding this comment.
The variable names old_path and new_path are confusing as they don't reflect the actual operation. The code moves from Source/Packages (new_path) to source/SRPMS (old_path), but semantically this is moving from the new format to the old format for backward compatibility. Consider renaming to backward_compatible_path and new_arrow_path for clarity.
| old_path = "#{distribution_path}/source/SRPMS" | |
| new_path = "#{distribution_path}/Source/Packages" | |
| mkdir_p(File.dirname(old_path)) | |
| mv(new_path, old_path) | |
| rm_rf(File.dirname(new_path)) | |
| backward_compatible_path = "#{distribution_path}/source/SRPMS" | |
| new_arrow_path = "#{distribution_path}/Source/Packages" | |
| mkdir_p(File.dirname(backward_compatible_path)) | |
| mv(new_arrow_path, backward_compatible_path) | |
| rm_rf(File.dirname(new_arrow_path)) |
There was a problem hiding this comment.
backward_compatible_path is easier to understand.
|
ここまで頂いたレビューコメントの対応を行いました。 |
| keep_srpm_paths_for_compatibility | ||
| remove_srpm_from_arm_architectures |
There was a problem hiding this comment.
Could you add yum_ prefix?
| keep_srpm_paths_for_compatibility | |
| remove_srpm_from_arm_architectures | |
| yum_keep_srpm_paths_for_compatibility | |
| yum_remove_srpm_from_arm_architectures |
| mv(new_path, backward_compatible_path) | ||
| rm_rf(File.dirname(new_path)) | ||
| else | ||
| # Use latest SRPM path for other distributions Source/Packages/*.src.rpm |
There was a problem hiding this comment.
| # Use latest SRPM path for other distributions Source/Packages/*.src.rpm | |
| # Use SRPM path as-is for other distributions: Source/Packages/*.src.rpm |
| end | ||
| end | ||
|
|
||
| def remove_srpm_from_arm_architectures |
There was a problem hiding this comment.
| def remove_srpm_from_arm_architectures | |
| def remove_srpm_from_non_primary_architectures |
|
Thank you for the reviews. I’ve addressed all the comments so far. |
Apache Arrow changed SRPM output location from
source/SRPMS/toSource/Packages/.This change breaks Yum repository compatibility for existed distributions. This commit adds path management after
yum:buildto maintain compatibility:Source/Packages/tosource/for backward compatibilitySource/Packages/path