Skip to content

package yum: keep SRPM paths for backward compatibility#2558

Merged
kou merged 15 commits intogroonga:mainfrom
otegami:rename-yum-srpm-path
Oct 15, 2025
Merged

package yum: keep SRPM paths for backward compatibility#2558
kou merged 15 commits intogroonga:mainfrom
otegami:rename-yum-srpm-path

Conversation

@otegami
Copy link
Contributor

@otegami otegami commented Oct 10, 2025

Apache Arrow changed SRPM output location from source/SRPMS/ to Source/Packages/.

This change breaks Yum repository compatibility for existed distributions. This commit adds path management after yum:build to maintain compatibility:

  • For AlmaLinux 10 and earlier and Amazon Linux 2023: Rename SRPMs path from Source/Packages/ to source/ for backward compatibility
  • For AlmaLinux 11 and later: Uses the new Source/Packages/ path
  • Except for x86 architecture: Removes SRPMs

@otegami otegami force-pushed the rename-yum-srpm-path branch 4 times, most recently from 82c5328 to 7c36b73 Compare October 10, 2025 10:57
Apache Arrow changed SRPM output location from 'Source/Packages/' to
'source/SRPMS/'.

This change breaks Yum repository compatibility for existing AlmaLinux
distributions. This commit adds path management after yum:build to
maintain compatibility:

- For AlmaLinux 10 and earlier: Rename source/SRPMS/ to Source/Packages/
- For AlmaLinux 11 and later: Uses the new source/SRPMS/ structure
- For aarch64 architecture: Removes SRPMs
@otegami otegami force-pushed the rename-yum-srpm-path branch from 7c36b73 to 5e0869f Compare October 12, 2025 07:50
@otegami
Copy link
Contributor Author

otegami commented Oct 12, 2025

Before this PR

$ tree packages-almalinux-8-x86_64.new/
packages-almalinux-8-x86_64.new/
├── almalinux-8-x86_64
│   └── packages
│       └── yum
│           └── repositories
│               └── almalinux
│                   └── 8
│                       ├── Source
│                       │   └── Packages
│                       │       └── groonga-15.1.8-1.el8.src.rpm
│                       └── x86_64
│                           └── Packages
│                               ├── groonga-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-debugsource-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-devel-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-doc-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-examples-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-libs-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-libs-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-munin-plugins-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-h3-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-suggest-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-server-common-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-server-gqtp-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-server-http-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-token-filter-stem-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-tokenizer-mecab-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               └── groonga-tools-15.1.8-1.el8.x86_64.rpm
└── almalinux-8-x86_64.tar.gz

11 directories, 23 files

After this PR

$ tree packages-almalinux-8-x86_64/
packages-almalinux-8-x86_64/
├── almalinux-8-x86_64
│   └── packages
│       └── yum
│           └── repositories
│               └── almalinux
│                   └── 8
│                       ├── source
│                       │   └── SRPMS
│                       │       └── groonga-15.1.8-1.el8.src.rpm
│                       └── x86_64
│                           └── Packages
│                               ├── groonga-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-debugsource-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-devel-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-doc-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-examples-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-libs-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-libs-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-munin-plugins-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-h3-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-suggest-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-server-common-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-server-gqtp-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-server-http-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-token-filter-stem-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-tokenizer-mecab-15.1.8-1.el8.x86_64.rpm
│                               ├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.x86_64.rpm
│                               └── groonga-tools-15.1.8-1.el8.x86_64.rpm
└── almalinux-8-x86_64.tar.gz

11 directories, 23 files

@otegami otegami marked this pull request as ready for review October 12, 2025 08:57

def yum_build(console: false)
super
manage_srpm_paths_for_compatibility
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manageってどういうニュアンスで使っていますか?
私は「なんとかする」くらいの意味だと思っています。もしそういうニュアンスで使っているなら、そういうふわっとした意味よりもっと明確な操作を示す単語を使って欲しいです。

keep_srpm_path_compatibilityとか?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ありがとうございます。
次の2つの作業をやらせていたので、manageを利用していました。
fix: 81066b0 ただここでは、互換性に関する処理だけを行いたいので、keep_xxに変更します。
fix: a70ed92 削除自体も別に切り出して対応します。

  • 互換性のためにパスを変更する
  • ARM64環境では、SRPMを削除する

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: e015dc0 2回同じループを回すのが引っかかったので、manage_srpm_pathsで互換性の処理とARM64環境の対応を合わせて対応させてみました。
(変更前のほうが何をするかすぐにわかりやすいかなとも思っているので少し悩んでいます。)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

遅い処理でもなく、速度が求められる処理でもないので、読みやすさを犠牲にするメリットがないです。そのため、一緒にしなくていいっす。

こういう読みやすさを犠牲にする最適化をするべきかどうかという判断をするときは、そもそも遅くて困っているのか?困っているならどのくらい速くしないといけないのかとかを考えるべきです。一般的には。

たとえば、こういう開発用のツールとか、そもそもループで回すものが少ないときは、最適化する必要がないことが多いです。一方、Groongaの全文検索関連の処理とかは速さがマジで重要で、扱うデータ量も多いので、読みやすさを犠牲にしてでもがんばらないといけないという判断が必要なときが多々あります。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: 22db6ba
ありがとうございます。今後、Groongaだけに閉じずに開発する際に意識していこうと思います!

Comment on lines 82 to 86
# Backward compatibility: only rename for existing distributions that
# still expect the legacy SRPM structure.
# Legacy (<= AlmaLinux 10): source/SRPMS
# New (>= AlmaLinux 11): Source/Packages (no rename needed)
next if distribution != "almalinux"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

この条件だとAlmaLinux 11でもリネームしませんか?

現状で対応しているディストリビューションが以下なので、これらはリネームし、それ以外は何もしない、がいいです。

def yum_targets_default
[
"almalinux-8",
"almalinux-8-aarch64",
"almalinux-9",
"almalinux-9-aarch64",
"almalinux-10",
"almalinux-10-aarch64",
"amazon-linux-2023",
"amazon-linux-2023-aarch64",
]
end

今の実装だとAmazonLinuxのことは考慮していないですよね。

Suggested change
# Backward compatibility: only rename for existing distributions that
# still expect the legacy SRPM structure.
# Legacy (<= AlmaLinux 10): source/SRPMS
# New (>= AlmaLinux 11): Source/Packages (no rename needed)
next if distribution != "almalinux"
# Rename SRPM directory only for the following distributions
# to keep backward compatibility:
#
# * AlmaLinux 8
# * AlmaLinux 9
# * AlmaLinux 10
# * Amazon Linux 2023
#
# They keep using source/SRPMS/*.src.rpm for SRPM path.
case "#{distribution}-#{version}"
when "almalinux-8",
"almalinux-9",
"almalinux-10",
"amazon-linux-2023"
next
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: af2933d 対応するディストリビューションを明示的に記載するように修正しています。

Comment on lines 87 to 89
repo_base = "#{repositories_dir}/#{distribution}/#{version}"
old_path = "#{repo_base}/source/SRPMS"
new_path = "#{repo_base}/Source/Packages"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

old_path/new_pathにするならrepo_baseXXX_pathにしませんか?

Suggested change
repo_base = "#{repositories_dir}/#{distribution}/#{version}"
old_path = "#{repo_base}/source/SRPMS"
new_path = "#{repo_base}/Source/Packages"
base_path = "#{repositories_dir}/#{distribution}/#{version}"
old_path = "#{base_path}/source/SRPMS"
new_path = "#{base_path}/Source/Packages"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: 15485bb pathの名前に関しては一貫して、xx_pathを利用するように修正しました。

@otegami
Copy link
Contributor Author

otegami commented Oct 14, 2025

ここまで頂いたレビューコメントの対応を行いました。
改めて期待通り動作しているのか、x86とaarch64環境で確認を行いました。

動作確認ログ
$ tree almalinux-8-x86_64/
almalinux-8-x86_64/
└── packages
    └── yum
        └── repositories
            └── almalinux
                └── 8
                    ├── source
                    │   └── SRPMS
                    │       └── groonga-15.1.8-1.el8.src.rpm
                    └── x86_64
                        └── Packages
                            ├── groonga-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-debuginfo-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-debugsource-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-devel-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-doc-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-examples-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-libs-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-libs-debuginfo-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-munin-plugins-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-plugin-h3-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-plugin-suggest-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-server-common-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-server-gqtp-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-server-http-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-token-filter-stem-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-tokenizer-mecab-15.1.8-1.el8.x86_64.rpm
                            ├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.x86_64.rpm
                            └── groonga-tools-15.1.8-1.el8.x86_64.rpm

10 directories, 22 files
$ tree almalinux-8-aarch64/
almalinux-8-aarch64/
└── packages
    └── yum
        └── repositories
            └── almalinux
                └── 8
                    └── aarch64
                        └── Packages
                            ├── groonga-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-debuginfo-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-debugsource-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-devel-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-doc-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-examples-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-libs-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-libs-debuginfo-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-munin-plugins-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-plugin-h3-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-plugin-h3-debuginfo-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-plugin-suggest-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-plugin-suggest-debuginfo-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-server-common-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-server-gqtp-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-server-http-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-token-filter-stem-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-token-filter-stem-debuginfo-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-tokenizer-mecab-15.1.8-1.el8.aarch64.rpm
                            ├── groonga-tokenizer-mecab-debuginfo-15.1.8-1.el8.aarch64.rpm
                            └── groonga-tools-15.1.8-1.el8.aarch64.rpm

8 directories, 21 files

- achitectures
+ architectures
@otegami
Copy link
Contributor Author

otegami commented Oct 14, 2025

fix: ce5b45b I will fix typos.

@otegami otegami force-pushed the rename-yum-srpm-path branch from f71e64c to 4625aee Compare October 14, 2025 02:56
@otegami otegami force-pushed the rename-yum-srpm-path branch from 4625aee to e015dc0 Compare October 14, 2025 03:08
This reverts commit e015dc0.

We don't have to persue the speed here because there
are not a lot of distibutions. Readability is more
important.
Comment on lines 104 to 105
else
next
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
else
next

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

明示的にコメントだけ入れるようにしてみました。
" # Use latest SRPM path for other distributions Source/Packages/*.src.rpm"

repositories_path = "#{yum_dir}/repositories"
yum_targets.each do |target|
distribution, version, architecture = split_target(target)
next if architecture != "aarch64"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

やりたいことはSRPMは1つだけ残して全部消したい、なので、aarch64だけ消す、じゃなくて、x86_64以外は消すというコードにしませんか?(オリジナルのコードも[ "${ARCHITECTURE}" != "x86_64" ]なのでそういう意図のコードになっていました。)

x86_64のときはarchitecturenilになる気がするのでnext unless architecture.nil?かな。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

やりたいことはSRPMは1つだけ残して全部消したい

fix: 2c13cb2 ありがとうございます。なるほどです。aarch64の場合だけ削除すると捉えてしまっていたのですが、本質的には何をやりたいのかちゃんと理解できていなかったですmm

@otegami
Copy link
Contributor Author

otegami commented Oct 14, 2025

ここまで頂いたレビューコメントの対応を行いました。

@abetomo abetomo requested a review from Copilot October 14, 2025 23:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors SRPM (Source RPM) path management from a shell script approach to Ruby methods to handle backward compatibility with different AlmaLinux distributions following Apache Arrow's SRPM location changes.

  • Moves SRPM path management logic from GitHub Actions workflow to Ruby methods in the package task
  • Implements distribution-specific path handling for AlmaLinux 8-10 and Amazon Linux 2023
  • Replaces shell-based SRPM removal for non-x86_64 architectures with Ruby implementation

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
packages/packages-groonga-org-package-task.rb Adds Ruby methods for SRPM path compatibility and architecture-specific removal
.github/workflows/package.yml Removes shell script logic for SRPM handling from the workflow

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment on lines 99 to 103
old_path = "#{distribution_path}/source/SRPMS"
new_path = "#{distribution_path}/Source/Packages"
mkdir_p(File.dirname(old_path))
mv(new_path, old_path)
rm_rf(File.dirname(new_path))
Copy link

Copilot AI Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable names old_path and new_path are confusing as they don't reflect the actual operation. The code moves from Source/Packages (new_path) to source/SRPMS (old_path), but semantically this is moving from the new format to the old format for backward compatibility. Consider renaming to backward_compatible_path and new_arrow_path for clarity.

Suggested change
old_path = "#{distribution_path}/source/SRPMS"
new_path = "#{distribution_path}/Source/Packages"
mkdir_p(File.dirname(old_path))
mv(new_path, old_path)
rm_rf(File.dirname(new_path))
backward_compatible_path = "#{distribution_path}/source/SRPMS"
new_arrow_path = "#{distribution_path}/Source/Packages"
mkdir_p(File.dirname(backward_compatible_path))
mv(new_arrow_path, backward_compatible_path)
rm_rf(File.dirname(new_arrow_path))

Copilot uses AI. Check for mistakes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backward_compatible_path is easier to understand.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: 96fc07b Thanks!

@otegami
Copy link
Contributor Author

otegami commented Oct 15, 2025

ここまで頂いたレビューコメントの対応を行いました。

Comment on lines 73 to 74
keep_srpm_paths_for_compatibility
remove_srpm_from_arm_architectures
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add yum_ prefix?

Suggested change
keep_srpm_paths_for_compatibility
remove_srpm_from_arm_architectures
yum_keep_srpm_paths_for_compatibility
yum_remove_srpm_from_arm_architectures

Copy link
Contributor Author

@otegami otegami Oct 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: 1ca7a60 Sure!

mv(new_path, backward_compatible_path)
rm_rf(File.dirname(new_path))
else
# Use latest SRPM path for other distributions Source/Packages/*.src.rpm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Use latest SRPM path for other distributions Source/Packages/*.src.rpm
# Use SRPM path as-is for other distributions: Source/Packages/*.src.rpm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: 162df30

end
end

def remove_srpm_from_arm_architectures
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def remove_srpm_from_arm_architectures
def remove_srpm_from_non_primary_architectures

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix: 4dfee33

@otegami
Copy link
Contributor Author

otegami commented Oct 15, 2025

Thank you for the reviews. I’ve addressed all the comments so far.

@otegami otegami changed the title package yum: manage SRPM paths for backward compatibility package yum: keep SRPM paths for backward compatibility Oct 15, 2025
@kou kou merged commit 31ab0f0 into groonga:main Oct 15, 2025
2 checks passed
@kou kou deleted the rename-yum-srpm-path branch October 15, 2025 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants