Skip to content

Revert "Add full_hostname host variable"#46478

Merged
haampie merged 2 commits intodevelopfrom
revert-45522-hostname-is-fqdn
Sep 19, 2024
Merged

Revert "Add full_hostname host variable"#46478
haampie merged 2 commits intodevelopfrom
revert-45522-hostname-is-fqdn

Conversation

@haampie
Copy link
Copy Markdown
Member

@haampie haampie commented Sep 19, 2024

It looks like this commit causes a major performance penalty in unit test time on macOS. Fine with the feature in principle, but has to be redone.

Reverts #45522

@spackbot-app spackbot-app bot added core PR affects Spack core functionality documentation Improvements or additions to documentation labels Sep 19, 2024
@haampie
Copy link
Copy Markdown
Member Author

haampie commented Sep 19, 2024

@Jordan474 I'm reverting this for now, since apparently 30 minutes of CI time is spent on getfqdn. Any clue where that overhead is coming from? Feel free to resubmit a PR.

@haampie haampie merged commit 586360a into develop Sep 19, 2024
@haampie haampie deleted the revert-45522-hostname-is-fqdn branch September 19, 2024 13:34
@tgamblin
Copy link
Copy Markdown
Member

We call socket.getfqdn() when writing the database, as well. It's used in the tempfile name. I wonder if that is also causing a performance penalty on macOS.

@Jordan474
Copy link
Copy Markdown
Contributor

Indeed, there are several reports of issues with fqdn on macOS.

There seems to be a user-side workaround: scutil --set HostName $(scutil --get LocalHostName). However, I did not find a definitive solution suitable for Spack yet.

I'm wondering if the hostname -f is affected too ? (its implementation differs from socket.fqdn). I'll look into github macos runners to test this.

Note that even on Linux, I probably should have memoized fqdn, because fqdn could query the DNS if /etc/hosts does not contain the non-fqdn hostname.

@Jordan474
Copy link
Copy Markdown
Contributor

Jordan474 commented Sep 24, 2024

I tested a few options on github latest runners:

ubuntu macos windows
hostname fast fast fast
hostname -f fast fast FAIL (1)
socket.gethostname() fast fast fast
socket.getfqdn() fast slow (5s) less fast (100ms)
socket.getfqdn(gethostname()[:-len(".local")]) (2) n/a fast n/a
scutil --get HostName n/a fast n/a

"fast" means between 1 to 10 ms. I removed python cold start and overheads before comparing the results.

(1): on the windows runner, hostname -f fails with unknown option -- f

(2): On macos, the full hostname is just hostname plus .local (Mac-1727169450078 -> Mac-1727169450078.local). On ubuntu and windows runners, full hostnames have way more components.

EDIT: To clarify, on macos, socket.gethostname() == socket.getfqdn() == scutil --get HostName == Mac-123.local, but scutil --get LocalHostName == Mac-123.


The .local name seems special enough, so maybe the simplest workaround would be to handle it explicitly:

@memoized
def _getfqdn():
    name = socket.gethostname()
    # Local hostname, avoid slow getfqdn on MacOS
    if name.endswith(".local"):
        return name
    return socket.getfqdn(name)

I don't think it's worth it implementing a different subprocess for each platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core PR affects Spack core functionality documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants