Add prototype pkgfilegroup-based RPM builder by nacl · Pull Request #129 · bazelbuild/rules_pkg

nacl · 2020-01-15T14:11:36Z

NOTE: only the third commit (currently bd8d531) should be reviewed as a part of this change. The first two should be reviewed as a part of #128.

This change provides a prototype pkgfilegroup-based RPM builder in the form of
the gen_rpm rule. See #128 for more details on pkgfilegroup.

The RPM generator was derived from make_rpm.py in pkg/ and supports a number
of features over and above what's available in pkg_rpm. As written, it, given
a template like the one provided, can construct many full-fledged RPM
packages entirely within Bazel. In most cases, the templates will only need to
be customized with advanced logic and other macros that are not settable via
bazel itself; gen_rpm will write much of the preamble, %description text,
%install scriptlets, and %files based on rule-provided inputs.

Documentation outside of the source files is not yet available. This was
empirically tested on RPM packages internal to VMware with positive results;
actual tests of the rules are not yet ready.

This, naturally, is incomplete, and is missing capabilities such as:

Configurable compression
Configurable Provides/Requires
SRPM emission
Reproducibility
Configurable stripping
Configurable construction of "debug" packages

googlebot · 2020-01-15T14:11:42Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to determine that you authored the commits in this PR. Maybe you used a different email address in the git commits than was used to sign the CLA? If someone else authored these commits, then please add them to this pull request and have them confirm that they're okay with them being contributed to Google. If there are co-authors, make sure they're formatted properly.

In order to pass this check, please resolve this problem and then comment@googlebot I fixed it... If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

googlebot · 2020-01-15T14:19:35Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

aiuto

Can you merge with the current upstream so this is a smaller change.

pkg/BUILD

nacl · 2020-02-28T16:56:08Z

Can you merge with the current upstream so this is a smaller change.

Done.

aiuto · 2020-03-11T21:56:21Z

Hi. I'm back from vacation, so I can take a look again.
There is a lot going on here. It seems two orthogonal things going on.

working with pkgfilegroup
general improvements to make_rpm.py itself

What are your plans to get the improvements migrated from experimental up to the top level package?
In the long run, we are going to need that.

pkg/experimental/make_gen_rpm.py

nacl · 2020-03-26T19:34:48Z

Hi. I'm back from vacation, so I can take a look again.
There is a lot going on here. It seems two orthogonal things going on.
1. working with pkgfilegroup

2. general improvements to make_rpm.py itself
What are your plans to get the improvements migrated from experimental up to the top level package?
In the long run, we are going to need that.

From what I can tell, it shouldn't be too difficult to combine the two together; the logic could be made more robust to multiple template types. Could also change the pkg_rpm starlark logic, but I think changes to make_rpm.py would be easier.

aiuto · 2020-04-13T19:52:49Z

What I was trying to get at was to not to do everything here and replace, but to take the rpm enhancements (like built in spec-file, pre/post scripts) and move them to the mainline code. Then have experimental derive from that.

But I'll be OK with getting this merged so that others can provide feedback no the pkg_filegroup stuff. While that is improving, we can backport the new features to the mainline code first, and bring in pkgfilegroup as we reach concensus.

aiuto

While we are still experimenting, can you name it pkg_filegroup, rather than pkgfilergroup so it matches the convention of all the other names.

pkg/experimental/genrpm.bzl

pkg/experimental/make_gen_rpm.py

nacl · 2020-04-14T13:41:12Z

While we are still experimenting, can you name it pkg_filegroup, rather than pkgfilergroup so it matches the convention of all the other names.

Certainly. How about I take care of this in a change that will go in after this one?

This change provides a prototype `pkgfilegroup`-based RPM builder in the form of the `gen_rpm` rule. See bazelbuild#128 for more details on `pkgfilegroup`. The RPM generator was derived from `make_rpm.py` in `pkg/` and supports a number of features over and above what's available in `pkg_rpm`. As written, it, given a template like the one provided, you can construct many full-fledged RPM packages entirely within Bazel. In most cases, the templates will only need to be customized with advanced logic and other macros that are not settable via bazel itself; `gen_rpm` will write much of the preamble, `%description` text, `%install` scriptlets, and `%files` based on rule-provided inputs. Documentation outside of the source files is not yet available. This was empirically tested on RPM packages internal to VMware with positive results; actual tests of the rules are not yet ready. This, naturally, is incomplete, and is missing capabilities such as: - Configurable compression - Configurable Provides/Requires - SRPM emission - Reproducibility - Configurable stripping - Configurable construction of "debug" packages Co-authored-by: mzeren-vmw <[email protected]> Co-authored-by: klash <[email protected]>

nacl · 2020-04-14T21:08:33Z

What I was trying to get at was to not to do everything here and replace, but to take the rpm enhancements (like built in spec-file, pre/post scripts) and move them to the mainline code. Then have experimental derive from that.

But I'll be OK with getting this merged so that others can provide feedback no the pkg_filegroup stuff. While that is improving, we can backport the new features to the mainline code first, and bring in pkgfilegroup as we reach concensus.

Ah, OK. Fair enough.

I'll work on this next after I clear out my stack of pending changes. I got symlink support and an RPM build tester in the pipe.

nacl · 2020-04-14T21:11:59Z

It just occurred me that I'm probably making reviews difficult by force-pushing updates all the time. Sorry about that. Here's the text diff for the latest change:

9e9e7a6-to-229a244.diff.txt

aiuto · 2020-04-15T22:21:59Z

One thing I truly hate about github reviews is that it is telling me I requested changes, and it won't point me to the place I asked for them. There are more things, but that is one.

While we are still experimenting, can you name it pkg_filegroup, rather than pkgfilergroup so it matches the convention of all the other names.

Certainly. How about I take care of this in a change that will go in after this one?

Sure.

This change renames all instances of `pkgfilegroup` with `pkg_filegroup`, and renames `experimental/genpkg.bzl` to `experimental/pkg_filegroup.bzl`. Names of tests and documentation have been updated accordingly. The change was mostly made mechanically using `sed(1)`, with minor cleanup afterward. This change makes the names within the experimental/ branch consistent, and was requested in the review of bazelbuild#129.

This change renames all instances of `pkgfilegroup` with `pkg_filegroup`, and renames `experimental/genpkg.bzl` to `experimental/pkg_filegroup.bzl`. Names of tests and documentation have been updated accordingly. The change was mostly made mechanically using `sed(1)`, with minor cleanup afterward. This change makes the names within the experimental/ directory consistent, and was requested in the review of bazelbuild#129.

This change renames all instances of `pkgfilegroup` to `pkg_filegroup`, and renames `experimental/genpkg.bzl` to `experimental/pkg_filegroup.bzl`. Names of tests and documentation have been updated accordingly. The change was mostly made mechanically using `sed(1)`, with minor cleanup afterward. This change makes the names within the experimental/ directory consistent, and was requested in the review of bazelbuild#129.

This change renames all instances of `pkgfilegroup` to `pkg_filegroup`, and renames `experimental/genpkg.bzl` to `experimental/pkg_filegroup.bzl`. Names of tests and documentation have been updated accordingly. The change was mostly made mechanically using `sed(1)`, with minor cleanup afterward. This change makes the names within the experimental/ directory consistent, and was requested in the review of #129.

nacl requested a review from aiuto as a code owner January 15, 2020 14:11

nacl force-pushed the topic/pkgfilegroup-full-rpmgen branch from 6c123bf to bd8d531 Compare January 15, 2020 14:19

MarkusTeufelberger mentioned this pull request Feb 3, 2020

Changed the buildroot from BUILD to BUILDROOT #108

Merged

aiuto requested changes Feb 28, 2020

View reviewed changes

pkg/BUILD Outdated Show resolved Hide resolved

nacl force-pushed the topic/pkgfilegroup-full-rpmgen branch from bd8d531 to 331cdc1 Compare February 28, 2020 16:53

nacl requested a review from aiuto February 28, 2020 16:55

aiuto requested changes Mar 11, 2020

View reviewed changes

pkg/experimental/make_gen_rpm.py Outdated Show resolved Hide resolved

nacl force-pushed the topic/pkgfilegroup-full-rpmgen branch from 331cdc1 to 9e9e7a6 Compare March 26, 2020 19:24

nacl requested a review from aiuto April 2, 2020 16:32

aiuto requested changes Apr 13, 2020

View reviewed changes

pkg/experimental/genrpm.bzl Outdated Show resolved Hide resolved

pkg/experimental/genrpm.bzl Outdated Show resolved Hide resolved

pkg/experimental/genrpm.bzl Show resolved Hide resolved

pkg/experimental/make_gen_rpm.py Outdated Show resolved Hide resolved

nacl force-pushed the topic/pkgfilegroup-full-rpmgen branch from 9e9e7a6 to 229a244 Compare April 14, 2020 20:57

nacl requested a review from aiuto April 14, 2020 21:01

aiuto approved these changes Apr 15, 2020

View reviewed changes

aiuto merged commit 1c799ba into bazelbuild:master Apr 15, 2020

nacl mentioned this pull request Apr 16, 2020

Mass pkgfilegroup -> pkg_filegroup rename #156

Merged

nacl mentioned this pull request May 11, 2020

Consolidate experimental and existing make_rpm scripts #173

Closed

nacl mentioned this pull request Sep 29, 2020

Generate spec file for RPM packages #241

Open

nacl deleted the topic/pkgfilegroup-full-rpmgen branch January 19, 2021 21:39

nacl mentioned this pull request Feb 23, 2021

Bugs/missing features in pkg_filegroup-using pkg_rpm #300

Closed

5 tasks

Conversation

nacl commented Jan 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

googlebot commented Jan 15, 2020

Uh oh!

googlebot commented Jan 15, 2020

Uh oh!

aiuto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nacl commented Feb 28, 2020

Uh oh!

aiuto commented Mar 11, 2020

Uh oh!

Uh oh!

nacl commented Mar 26, 2020

Uh oh!

aiuto commented Apr 13, 2020

Uh oh!

aiuto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nacl commented Apr 14, 2020

Uh oh!

nacl commented Apr 14, 2020

Uh oh!

nacl commented Apr 14, 2020

Uh oh!

aiuto commented Apr 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nacl commented Jan 15, 2020 •

edited

Loading