Ok good news and bad news.

First the good news. Still no diffs with MPI or GPU. On GPU, with the challenge problem, we significantly improved the initialize_eb2_structs:
this PR:

PeleC::initialize_eb2_structs()                     51      0.211      0.211      0.211   0.30%

development branch:

PeleC::initialize_eb2_structs()                     51      11.74      11.74      11.74  19.92%

So yay.

The bad news. The apply_face_stencil blew up:
This PR:

PeleC::pc_apply_face_stencil()                     800      23.41      23.41      23.41  33.80%

development branch:

PeleC::pc_apply_face_stencil()                     800      1.004      1.004      1.004   1.71%

I think what's going on is that I have a bunch of duplicate faces and I am not catching those.

Ok I think I figured it out.

On the challenge problem, the apply face stencil goes back to being:

PeleC::pc_apply_face_stencil()                     800      1.067      1.067      1.067   2.28%

which is basically the same as development.
And the init eb structs drops to:

PeleC::initialize_eb2_structs()                     51    0.07289    0.07289    0.07289   0.16%

which is  a 160X speedup for that function.

I am getting unexpected diffs on GPU with the challenge problem. I don't see them on CPU on my laptop...