Skip to content

move depfile logic into its own module, separate traversal logic from model#36911

Merged
haampie merged 7 commits intospack:developfrom
haampie:fix/depfile
Apr 17, 2023
Merged

move depfile logic into its own module, separate traversal logic from model#36911
haampie merged 7 commits intospack:developfrom
haampie:fix/depfile

Conversation

@haampie
Copy link
Copy Markdown
Member

@haampie haampie commented Apr 14, 2023

The depfile logic was more of a quick and dirty script in cmd/env.py, this PR
extracts the traversal and model stuff into its own module, so it gets easier
to write tests for it.

@spackbot-app spackbot-app bot added commands core PR affects Spack core functionality environments labels Apr 14, 2023
@alalazo
Copy link
Copy Markdown
Member

alalazo commented Apr 14, 2023

Very nice, I can follow with a smaller one to remove some ugliness from spack.bootstrap

@haampie
Copy link
Copy Markdown
Member Author

haampie commented Apr 14, 2023

@scheibelp you could now add a test along the lines of

import spack.environment.depfile as df
from spack.spec import Spec

x, y, z = Spec("x"), Spec("y"), Spec("z")

model = df.MakefileModel(
    env_path="somewhere",
    roots=[x],
    adjacency_list=[
        df.DepfileNode(x, [y], df.UseBuildCache.AUTO),
        df.DepfileNode(y, [z], df.UseBuildCache.AUTO)
    ],
    make_prefix="test",
    pkg_identifier_variable="SPACK_PACKAGE_IDS",
    jobserver=False,
)

data = model.to_dict()

assert ... data["..."] ...

@scheibelp
Copy link
Copy Markdown
Member

This is a version of the initial implementation I suggested in #36642 (comment), and has the same issue I described there: The unsafe nature of general spec formatting is not encoded into this solution and so it is easy to introduce special characters as part of spec.format or str(spec) (in spite of the _safe_name function you added here). In other words, if this were to merge, I would want the test I wrote in #36642 as it is currently written (validating the final makefile rather than an intermediate object).

You haven't directly responded to the follow-up suggestion I made in that comment - could you do that? Overall I like these changes, but don't find them specifically relevant to the problem that #36642 attempts to address.

Copy link
Copy Markdown
Member

@alalazo alalazo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically LGTM, a few minor comments. With this I think I can clean up:

spackcmd = spack.util.executable.which("spack")
spackcmd(
"-e",
str(self.environment_root()),
"env",
"depfile",
"-o",
str(self.environment_root().joinpath("Makefile")),
)

alalazo
alalazo previously approved these changes Apr 17, 2023
@haampie haampie merged commit a676f70 into spack:develop Apr 17, 2023
@haampie haampie deleted the fix/depfile branch April 17, 2023 13:27
haampie added a commit that referenced this pull request Apr 17, 2023
haampie added a commit that referenced this pull request Apr 17, 2023
haampie added a commit to haampie/spack that referenced this pull request Apr 18, 2023
RikkiButler20 pushed a commit to RikkiButler20/spack that referenced this pull request Apr 25, 2023
RikkiButler20 pushed a commit to RikkiButler20/spack that referenced this pull request Apr 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

commands core PR affects Spack core functionality environments

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants