From Bob G:

I'm now largely successful in the base suite on theia with intel -- 99 pass, 6 fail. The bulk of the problem was compiler options -- though I copied a file labeled intel, it was actually gnu options.

One of the failures is the 40x4 roundrobin, which I think is invoking a problem we have long had with CICE here (theia/ NOAA R+D) -- model getting in to a state where it isn't writing output, but also isn't crashing gracefully. I killed it after an hour of no change.

Do we have any way of collecting stats on which tests are most likely to pass or fail? How long they take? It'd be convenient to put a fast, frequently failing, test up front in the queue when we're to a point of requiring all to pass.