Still a work in progress. I tried testing on my local machine, but building CMake 3.5.2 hangs with gcc 6.1 on OS X 10.10.5. I forget if this issue with CMake is known re: OS X. If it is new for Spack, I will add it to the GitHub issue tracker.

Note to self: supposedly CMake 3.6.0 builds fine with gcc 6.1, so if there's been no progress on that when I pick this thread back up, upgrade the CMake package in spack.

@tgamblin I just rebased this PR on the develop branch last night and still got the same object.h error with CMake when building with gcc 6.1 due to incompatible headers shipped with OS X 10.10. How did you fix that again? Just build with clang for now?

I added CMake 3.6.1 in #1412. You could try that. From reading #1203, it sounds like you'll either have to patch that system header file or upgrade to a newer version of macOS.

@adamjstewart I'd be happy to add that patch to Spack, if there's a high likelihood it would be accepted. I think I would modify it according to http://stackoverflow.com/a/28014302/1102947 so the code remains practically unchanged for clang. Upgrading macOS at LLNL would be more difficult, but possible.

I would accept that patch to gcc and clang. If you want to add it, that would be great. It wouldn't fix your issue if you used an external compiler, but it would if you used spack-built ones.

@tgamblin Having fought through most of the errors due to gcc incompatibility with Apple headers during CMake installation, I think the only hangup in this PR right now (aside from any flake8 noncompliance) is the hack to go through the mfem URL tracker. The strategy that worked for mfem 3.1 does not seem to work for mfem 3.2. Replacing the tracker URL with its redirected URL fixes this issue, and also makes the spack fetch/stage/build process much, much less fragile.

@tzanio, how important is it to have spack fetch mfem tarballs through the Google redirect? I realize the redirect is valuable for user metrics, and Homebrew seems to cope with it just fine, but I'm having trouble making it work for spack, and the redirect issue is the major blocker for this pull request.

@goxberry, thanks for adding these!

The stats have been really useful for us, including in some pretty major proposals, so if possible we'll prefer to keep them. Is it clear what exactly the problem is? Maybe there is an easy way to fix that in spack?

If the fix is not obvious, we can move on with the direct links. I imagine we are not the only project that is interested in tracking spack installations so maybe @tgamblin and team have a different solution?

@tgamblin, did you get a chance to look at this?

@tzanio I think part of the issue is that Spack will download archives and store them in a location that uses the base name of the URL; this base name is Y9T75B for mfem 3.2 and xrScXn for mfem 3.1. The URL lacks both the software package name ("mfem") and an extension for decompressing it (here, it would be ".tar.gz"). I worked around the latter issue by explicitly decompressing the archive in lines 181 to 186 of the pull request version of mfem/package.py, but I think the former is what causes issues with Spack's otherwise extremely helpful autodetection heuristics.

I'm speculating, but I wonder if there were an option to manually specify the download file (so I could call the 3.2 tarball, mfem-3.2.tar.gz, and the 3.1 tarball, mfem-3.1.tar.gz), would that solve some of these issues?

I fixed a gcc bug that was blocking builds of cmake from source, so now I think the main thing blocking updates of the mfem spack package is this PR, and solving the downloading issue.

cc: @tgamblin

@goxberry: I was about to write an explanation for this the other day, but I got more email. Sorry. 😢

We can actually get the name of the remote tarball when we download from the internet -- newer versions of curl have a -J option that grabs the Location header from the redirect response and saves the file as the basename of the Location URL. RHEL6 (and therefore lots of HPC clusters) have a curl that is too old for that, but would actually be pretty easy to implement manually in Spack -- we already grab MIME headers to try to warn users about things like a web proxy sending HTML instead of a tarball.

However, that won't work because we not only need to be able to fetch from the internet, but also from mirrors, and Spack needs to be able to come up with the mirror name without talking to the redirect server, since it won't be there when we're not connected to the internet. Right now, we change mirror archive names to <package>-<version>.<extension>. The package and version are available statically (we know what we're fetching), but the extension isn't. We preserve the extension and compression method so that the checksums stay the same on the internet and in the mirror. With the google URLs, we don't know the extension, so you end up in this situation where you need to use expand=False because Spack doesn't otherwise know what to do with the file to get it into the mirror.

It turns out @scheibelp already thought of all this in #1758 w.r.t. #1757, and I didn't actually make the connection. I think I'll just merge that, and then we can make MFEM work more naturally.

@tgamblin No worries! With all the activity on the Spack repo, I'm amazed you keep up with everything, and I continue to be impressed by Spack. There's a whole bunch of CMake stuff I need to do on my end to eventually be able to use the MFEM package, but my hope is that it'll go into a couple LLNL projects, and then when that happens, I'll forward you the details via e-mail for any reporting you need to do for Spack funding. I've put a reminder in my calendar for September 6th, 2017 to remind me before end of FY17, because it might take that long to do all the CMake stuff so I can use Spack's CMake interfacing capabilities. (Er, build toolchains can be hard, especially when converting someone else's to CMake...) If you have an earlier deadline after, say January/February 2017, let me know, I'll move the reminder to that date.

@goxberry: can you see if you can get this working now that #1758 is merged?

@goxberry: I think @citibeth might be interested in the CMake stuff as well -- she wrote most of it, but it would be awesome to get some MFEM support in. See also #1926 if that is something you're thinking of... would like to replace some of the nextgen guys' env scripts with something like bundler.

@goxberry If you're looking to conveniently develop a project with Spack+CMake, see the documentation section:

https://spack.readthedocs.io/en/latest/workflows.html#developing-software-with-spack

Good luck, and don't hesitate to ping me if you run into something.

@tgamblin Yes, it works. I've done some commit cleanups so that every commit is executable (no syntax errors).

cc: @tzanio I think this PR is close to mergeable, so mfem 3.2 will hopefully be in Spack soon.

@citibeth I saw that, and I'm really impressed by the Spack + CMake support, so I am planning on trying it out on some LLNL projects I work on that have many developers. I'm hoping that a more automated install process will help new developers on these projects get up to speed faster, with less hassle, and also make the build toolchain more portable and maintainable.

@goxberry Wow that's nice to hear; definitely shout out if you need help. Just one caveat... it will work best if each developer has their on Spack instance that they own. I'm just beginning to broach the issues of how this stuff works on a shared multi-user environment.

@citibeth Yes, my goal is to make sure that everyone has their own Spack instance, and develops their own copy of the software we're working on. I will keep that advice in mind if someone pushes for multi-user environments.

@tgamblin Should I squash these commits?

@goxberry: no need. GitHub lets you choose to squash, merge, or rebase merge when you click the merge button.