@citibeth -- I'd like to see node and npm happen. Are you still interested in this? Can I help, and if so can you summarize the state of the pr?

@citibeth @hartzell I am not so sure we want to actually handle npm packages at the Spack level, and I'm not convinced that installing npm actually conflicts with Spack.

npm actually has some commonality with Spack in the way things are versioned and the way dependencies are managed, but there are also significant differences:

1. In an npm build, you can safely have two versions of the same package in the same DAG. The Javascript dependency model is thus very different from Spack's model, which is based on how most runtime linkers (ld.so, dyld, etc.) work.
2. npm packages are not "curated" like Spack packages -- npm is a service with a registry and there isn't a central approval model or any real curation. For that reason, npm and node move way faster than Spack can at this point. If Spack's package model gets more stable in the long-term, we could probably consider doing this, but right now I think we want to stay closely tied to packages as we're experimenting with how to write them well.
3. npm is an ecosystem in its own right and has something like 350,000 really fine-grained packages (something @citibeth might like!).
4. I am no js expert, but I think the installation model for node packages is rather different from how we do things in Spack.

So, based on this stuff, and the 350k package number, I don't think Spack should tackle wrapping the entire npm ecosystem. I certainly don't want to wrap 350K packages. I also don't think this is quite the same as pip. For Python, we actually do care about how underlying native libs are compiled. For npm, we might care about how node itself is compiled, but I don't think we care about how the npm packages are built from there.

So, I'm inclined to put node and npm in as regular packages the way @hartzell recommends. FWIW, this is how Homebrew does it -- allow npm to be installed and let it manage the js packages. This might seem inconsistent, but I think that packaging package managers needs to be a a case-by-case decision depending on:

1. The need for fine-grained control over a particular software ecosystem's build (IMHO, no need to do this for node);
2. The ability of Spack's dependency model to handle the ecosystem (It actually doesn't handle node's dependency model); and
3. Our ability to track the ecosystem (node is way too big to wrap)

So basically I agree with @hartzell on this one -- @hartzell @citibeth: opinions?

@tgamblin -- I wasn't arguing for anything in particular [yet], just wanted to understand the status of this PR.

@citibeth said:

the node.js stuff looks like "more of the same": AFAIK, it's a language-specific package manager that conflicts with Spack.

I rather think that Spack conflicts with it. They're a large and successful community that has a toolset that apparently works for them. If they want to live with things like the leftpad mess, I'm not going to bother to try to fix them. There are upsides to their infra-choices too...

My question is, what's the best way that I can support my developers who buy into that world, and what's the best way that I can install apps that require it.

This is evoking all of the same thoughts that our Perl and CPAN and cpanm conversation progressed through. It is useful to me to be able to use Spack to build and install the tools that underpin or bootstrap an ecosystem. I don't think that Spack can, or should, try to subsume all possible ecosystems, replacing their presumably evolutionarily fit tooling with something else. I certainly don't want to sign up to support those something-elses.

I would like, and do work towards, using tools like Spack to enable repeatable, well-behaved deployments. But there are languages and development practices that make it more trouble than it's worth (to me, at least).

I'd be happy to adopt this PR, get it to the point where it builds and tests and ... and see what that leaves us with.

I've never adopted a pull request before. Is there a well-principled way to get the branch into my own world and not loose the connection with the PR?

I said:

But there are languages and development practices that make it more trouble than it's worth (to me, at least).

More accurately (less confrontationally...), when I'm working with an "annoying" language but need reproducibility I've been solving it outside of tools like Spack, e.g. bootstrapping a perl and cpanm, using those to install some Perl specific tooling like Pinto or Carton and using that to maintain my own vendor'ed copy of the dependencies.

@hartzell: I think we're on the same page here. I think viewing this as an ecosystem bootstrapping process is the right way to go. I would love to have one package manager to rule them all but I don't think I would like to maintain it 😄.

I added you as a collaborator to Spack, so you can simply push to the features/npm_node_js branch in the Spack repo. Contributors with write access can edit any branches in Spack, but can't merge to master or develop (so you can blow up a branch but you can't blow up the community 💥).

Closing now that #2320 has been merged. Reopen if those packages aren't sufficient or need tweaks.