Skip to content

Conversation

@mwilck
Copy link
Contributor

@mwilck mwilck commented Dec 8, 2017

After discussion with @fbuihuu and @hreinecke, I attempted a rewrite of the udev rules for persistent storage, with the following goals:

  1. get rid of scsi_id, replace by sg calls / sysfs everywhere
    (this implies changes in 55-sg3-utils.rules as well)
  2. cleanly separate hardware detection, content detection (blkid), and
    symlink creation
  3. Improve general readability. For my taste, that means shorter
    lines, thus less conditions individual rules, thus more GOTOs
    separating blocks of code where certain conditins are known to
    hold.

This pull request needs to go together with the corresponding request for sg3_utils: hreinecke/sg3_utils#22

@yuwata yuwata added udev ci-fails/needs-rework 🔥 Please rework this, the CI noticed an issue with the PR labels Dec 9, 2017
# along with systemd; If not, see <http://www.gnu.org/licenses/>.

rules = files('''
55-storage-detection.rules
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

55-strage-hardware.rules

# dmsetup installs 55-dm and 60-persistent-storage-dm on Debian/Ubuntu
# see https://anonscm.debian.org/cgit/pkg-lvm/lvm2.git/tree/debian/patches/0007-udev.patch
inst_rules 55-dm.rules 60-persistent-storage-dm.rules
inst_rules 55-dm.rules 55-storage-hardware-rules \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong file name again...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

upps big sorry. I really tested this, no idea how that slipped in.

@mwilck
Copy link
Contributor Author

mwilck commented Dec 9, 2017

@yuwata, thanks for your thorough review. I fixed the issues you mentioned.

@yuwata yuwata removed the ci-fails/needs-rework 🔥 Please rework this, the CI noticed an issue with the PR label Dec 9, 2017
@yuwata
Copy link
Member

yuwata commented Dec 10, 2017

Hmm... I think removing scsi_id is drastic change. It may be used somewhere. Rather than completely 'kill' it, I prefer that scsi_id outputs a deprecate message and announce that somewhere, e.g. NEWS.

@yuwata
Copy link
Member

yuwata commented Dec 10, 2017

I am not sure that it can be comparable, but udevadm hwdb command is deprecated few years ago, but it still left there for compatibility.

@yuwata yuwata added the reviewed/needs-rework 🔨 PR has been reviewed and needs another round of reworks label Dec 10, 2017
@hreinecke
Copy link
Contributor

I was about to comment about the same thing; my idea for this whole conversion is to move all calls to scsi_id out of 60-persistent-storage.rules (like Martin did), but keep it in a rule set which complements the sg3_utils rules. The overall idea would be that one could keep one or other (or both or, of course, none); we would then have a series like

55-scsi-sg3_id.rules
58-scsi-sg3_symlink.rules
59-scsi-scsi_id.rules
60-persistent-storage.rules

And one could leave out the first two (if eg sg3_utils is not installed), or the 59 rules (if and when scsi_id gets deprecated), and still get the same user experience.

@mwilck
Copy link
Contributor Author

mwilck commented Dec 11, 2017

@yuwata, I had the impression that removing scsi_id was a set goal. One reason was the line in the TODO file that my patch removes, another remarks from @poettering I'd seen (e.g. his first comment on #7190), as well as various discussions with @hreinecke and @fbuihuu. I'm also quite positive that there are hardly any users of scsi_id outside udev/systemd.

However, if you prefer to keep it around, I've no problem with dropping the last patch in the series.

@mwilck
Copy link
Contributor Author

mwilck commented Dec 11, 2017

I'm sick today, I'll look into the build/packaging failures in detail later.
Obviously the additional rules files may cause packaging errors. They will also require changes in dracut.

@fbuihuu
Copy link
Contributor

fbuihuu commented Dec 11, 2017

OTOH scsi_id lives in /usr/lib/udev so it's not supposed to be used by anything but udev.

@poettering
Copy link
Member

dropping scsi_id makes a lot of sense to me, but i figure @kaysievers should review this before we merge this.

@mwilck
Copy link
Contributor Author

mwilck commented Dec 19, 2017

@hreinecke, I'm not sure I understand your proposal correctly. It's crucial to separate

  1. determination of device properties and creation of udev environment variables
  2. symlink creation based on environment variables
  3. further processing based on environment variables

After applying my patch set, the job of 60-persistent-storage.rules is 2.) exclusively - creation of symlinks based on certain generic environment variables which all follow the ID_xxx naming convention. We don't care any more by which utility these ID_xxx variables had been set before. It could be scsi_id, sg_inq, or whatever else. That's a big step forward.

This distinction should be mapped to rule numbers, too. In particular, 60 should be the separator between 1.) and 2.) above. IOW, SYMLINK calls should start only after 60. The rationale is simple - be sure all variables are correctly set before starting to set symlinks (there are a few exceptions for device-mapper rules currently, which we may want to address at a later stage).

It's also important to split step 1) above into hardware property detection an content-based property detection, mostly because we need to make "further processing" decisions between these two steps (think multipath). We need a distinction by number here as well, and as multipath is currently at 56, I chose that. Unfortunately, number space is becoming a bit tight between 55 and 60.

If we decide to keep scsi_id calls for setting ID_xxx variables, these rules should be placed after 55-scsi-sg3_utils.rules and before 56-multipath.rules. That's a bit tricky but it can be done.

58-scsi-sg3_symlink.rules is a bit special. It creates lots of symlinks which are not normally used by systemd or other tools which aren't specific to SCSI. I wonder if anyone except SUSE is using it.

@mwilck
Copy link
Contributor Author

mwilck commented Jan 19, 2018

There has been no progress for 4 weeks, any suggestions?

mwilck added a commit to openSUSE/multipath-tools-pre2021 that referenced this pull request Jan 19, 2018
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
mwilck added a commit to openSUSE/multipath-tools-pre2021 that referenced this pull request Jan 19, 2018
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
mwilck added a commit to openSUSE/multipath-tools-pre2021 that referenced this pull request Jan 19, 2018
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
@mwilck
Copy link
Contributor Author

mwilck commented Jan 24, 2018

Rebased, and modified the patch set as follows:

Instead of instantly removing the calls to scsi_id, I move them to a separate, new rule file "55-zz-scsi_id.rules". This means the rules using scsi_id are still in place. So distributions or users can choose using these or using something else (I would recommend sg3_utils). The scsi_id rules are only run if nothing else has set ENV{ID_SERIAL}.

The last patch removes the new rules file again, together with scsi_id itself. The idea is to give systemd maintainers the chance to wait a bit until killing of scsi_id, and open a migration path, by applying this series without the last patch. This also unties the dependency to the sg3_utils pull request hreinecke/sg3_utils#22 (as long as scsi_id rules are in present, no replacement is necessary).

Furthermore, I merged together the commits affecting the same device classes. I think review is easier this way, as you can see better how code is moved.

The udev rules from this patch set have been tested with various SCSI devices including disks, tapes, and medium changers, both using the scsi_id rules (all but last patch applied) and using the sg3_utils rules. I also verified that sg3_utils rules take precedence if both are installed. More testing is of course always welcome.

@mwilck mwilck changed the title udev rules: sanitize persistent-storage.rules, and get rid of scsi-id SCSI udev rules: separate out HW detection, prepare removal scsi-id Jan 26, 2018
@mwilck mwilck changed the title SCSI udev rules: separate out HW detection, prepare removal scsi-id SCSI udev rules: separate out HW detection, prepare removal of scsi-id Jan 26, 2018
mwilck added a commit to openSUSE/multipath-tools that referenced this pull request Sep 10, 2021
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
mwilck added a commit to openSUSE/multipath-tools that referenced this pull request Sep 15, 2021
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
mwilck added a commit to openSUSE/multipath-tools that referenced this pull request Dec 2, 2021
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
mwilck added a commit to openSUSE/multipath-tools that referenced this pull request Dec 6, 2021
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
mwilck added a commit to openSUSE/multipath-tools that referenced this pull request Dec 22, 2021
multipath -u needs ID_WWN für NVMe devices to determine the WWID.
ID_WWN is currently set after multipath.rules is processed.
Patches are under way to fix that
(systemd/systemd#7594).

Until then, insert a fallback rule here to make sure ID_WWN is
set when multipath is called.
facebook-github-bot pushed a commit to facebookincubator/antlir that referenced this pull request Jul 26, 2022
Summary:
Copy missing helper binaries from fedora to {,imaging_}ramdisk.

Note: this will need to be tweaked after systemd/systemd#7594 lands, maybe it's better to copy whole /usr/lib/udev?

Reviewed By: pallotron

Differential Revision: D38072013

fbshipit-source-id: a2c3b6ff6f2a9b4011ab544b1d08e445f42d5f25
@alexz-kh
Copy link

alexz-kh commented Jun 21, 2023

So, 5+ years, issue still appears on Supermicro X10DDW-iN + Intel® C612 chipset.
any chance to catch it ?

@mwilck
Copy link
Contributor Author

mwilck commented Jun 26, 2023

So, 5+ years, issue still appears on Supermicro X10DDW-iN + Intel® C612 chipset. any chance to catch it ?

What issue are you referring to concretely?

@alexz-kh
Copy link

@mwilck referenced one #7157
tl;tr : same ID_PATH reported by udev

@mwilck
Copy link
Contributor Author

mwilck commented Jun 26, 2023

see #7157 (comment)

@alexz-kh
Copy link

@mwilck missed by me! thanks. will reply to origin

@mauri
Copy link

mauri commented Aug 12, 2023

@mwilck Any chance to see this merged in the near future? I see this issue over and over in Ubuntu where sg3_utils is present and that breaks 60-persistent-storage rules, preventing ata_id properties to be imported. This have some nasty implications around disks APM capabilities not detected downstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

9 participants