@ChrisBarker-NOAA As for you code -- is this it:
https://github.com/pp-mo/ugrid-checks/tree/main/lib/ugrid_checks
Maybe this organization could host it and others can contribute.

That is where I'm working, but it's really only a start.
I'll certainly consider sharing here once I've got something at least halfway complete.
However, I think actually the ideal place would be in the cf-checker codebase -- but I haven't yet even begun a conversation about that.

@ChrisBarker-NOAA
Maybe you could put a big "DRAFT" note at the top, and then we could merge it. It would be nice to have it published for easier review and comment.

You're right, it is very much a draft at this point.

Also, I see as I work into the first-draft checker code, that I'm not even done making changes here (I already see I may need to add some statemnets, so I haven't really got to the the first version)
so this PR should probably be a Draft PR or WIP at this point.

Thanks so much for looking anyway, @ChrisBarker-NOAA
Should I should convert this PR to Draft for the time being ?

(I already see I may need to add some statemnets, so I haven't really got to the the first version)
so this PR should probably be a Draft PR or WIP at this point.

I'll put this in draft for now, just to be clear.
I hope to get somewhere definite with the checker code in the next few days.
After that, I'll mod this as needed, and re-submit it with a first proposal of the all the needed statements and number codes for them -- just because, beyond the initial version, the numbers will start to depart from a neat sequence.

Ok, I finally got around to some serious work on https://github.com/pp-mo/ugrid-checks/tree/main/lib/ugrid_checks
( where I now have a checker working + almost complete )
As predicted, that prompted some changes here, which I just pushed.

I'm now confident that this set of rules is workable (from the checker POV), so I took it out of draft.

My remaining concern here is to clearly label this "DRAFT", as @ChrisBarker-NOAA suggests.
I thought the nicest way would be a watermarking, so had a quick try at implementing the solution described here on StackOverflow, but my understanding of web tech is very thin and I have so far failed (!)
It's possible that mkdocs is in the way, but more likely IMHO that I don't understand CSS.
Any ideas ?
Anyway, I am back at my day job next week, where there is someone I can ask about these things, so I will try to pursue the watermarking idea that way.

Ok, I think I'm happy with this now.

FYI I tried a bunch of ideas for 'watermarking', but none of them interacted predictably with markdown/mkdocs, so I've gone with a simpler approach which totally avoids CSS (as you suggested @ChrisBarker-NOAA !)

@ChrisBarker-NOAA Maybe you could put a big "DRAFT" note at the top, and then we could merge it. It would be nice to have it published for easier review and comment.

Regarding that, if you merge does this automatically appear on the website itself ?

does this automatically appear on the website itself ?

OK I think I now understand that it does, via Travis ?

yes, I think they auto-build -- though I don't know that it's been tested in a while, so we'll see :-)

@core: I think it's a good idea to merge this, but I'd like at least one other person to agree. If any of you do agree, go ahead and merge.

Hi @ChrisBarker-NOAA
just FYI, I have just now published a installable "beta" release of my UGRID conformance checker utility based on these rules : https://github.com/pp-mo/ugrid-checks
-- please feedback !

As it is pretty functional now (I have been actually using it!), I wanted to share it for internal use.
So I just made it buildable+installable ; added basic usage instructions in the README ; tagged v0.1 and pushed to PyPI .
( I also particularly wanted to get out something useable, so that I can concentrate on other matters again for a while. )
We can maybe discuss that "living" in this organisation, too. I hope I haven't made dev tooling too involved for collaborators, but if so I'd be open to simplifying it.

As it made sense for those README usage instructions to refer to a list of conformance rules, just for now I have linked that account to a private RTD build including the current conformance page.
So, I hope that doesn't prejudice the principle that it is not agreed here yet !!

In view of @davidhassell recent progress on cf-convention/cf-conventions#353, it would be nice to come to some at-least-interim agreement on the conformance content here.
It also would be really good to resolve the question of the 3d/volume support, since that is another obstacle to adoption in CF.
I think I should be able to find some contacts here at the MetOffice who could comment on a possible "function space" encoding, as mentioned in relation to Fluidity/ICOM in that same discussion. For us that is still only a vague future possibility, I think, but I will see if anyone can be persuaded to comment!

Does it seem like we can address these things now, or do you see a better (future) time coming ?

@pp-mo: This looks really great! I've taken a quick look at your code, and it looks really nicely structured. tested, etc.

I haven't tried it much -- only one small example, file, but it worked well on that one :-)

Coverage of check,py is %80 - not perfect, but pretty good.

Name                                                                        Stmts   Miss  Cover
-----------------------------------------------------------------------------------------------
/Users/chris.barker/Temp/ugrid-checks/lib/ugrid_checks/__init__.py              4      0   100%
/Users/chris.barker/Temp/ugrid-checks/lib/ugrid_checks/check.py               803    166    79%
/Users/chris.barker/Temp/ugrid-checks/lib/ugrid_checks/nc_dataset_scan.py      64      6    91%
/Users/chris.barker/Temp/ugrid-checks/lib/ugrid_checks/scan_utils.py           27      0   100%
/Users/chris.barker/Temp/ugrid-checks/lib/ugrid_checks/ugrid_logger.py         70     10    86%
__init__.py                                                                    72      0   100%
check/__init__.py                                                               0      0   100%
check/test_check_dataset.py                                                   746      1    99%
nc_dataset_scan/__init__.py                                                     0      0   100%
nc_dataset_scan/test_scan_dataset.py                                           57      0   100%
-----------------------------------------------------------------------------------------------
TOTAL                                                                        1843    183    90%

As to where to go now:

If you think it's ready -- maybe we should make a conda=forge package for it - a lot of u use conda for netcdf works, etc.

We can maybe discuss that "living" in this organization, too.

I support that -- makes all the sense in the world

I hope I haven't made dev tooling too involved for collaborators, but if so I'd be open to simplifying it.

I didn't see anything scary :-)

it would be nice to come to some at-least-interim agreement on the conformance content here.

The challenge is folks finding the time to review it -- I gave it a quick once over and didn't see anything to complain about.

So maybe we can ping the group ( I wonder if the old email list is still functional?) to ask for review.

But I'm a believer that something mostly done is better than nothing :-) -- after all, the UGRID spec was a draft for a very long time. So even if we can't get much review, I'd be happy to see it merged in as a draft.

It also would be really good to resolve the question of the 3d/volume support, since that is another obstacle to adoption in CF.

Well, I don't know of anyone trying to use UGRID for 3-D volumes, and I haven't seen anyone step up to tackle it, so I think we can simply say that 3D support is tentative and not CF compliant.

Does it seem like we can address these things now, or do you see a better (future) time coming ?

Now is better than never -- let's do it!

(barring any objections from anyone else in the group)

@hrajagers, @rsignell-usgs, @ocefpaf : any thoughts?

@ChrisBarker-NOAA thanks so much for your attention + thoughtful discussion.

If you think it's ready -- maybe we should make a conda=forge package for it

I think I can easily sort that out. We do use conda use here mostly anyway, but it seemed that pip/PyPI was jusy a bit quicker for minimal effort to get an initial release out. I noted that the structure of conda recipes has shifted a bit since I last handled one (only a few months ago!), but I am lucky to have other people in my group who do have a good up-to-date knowledge, so I hope to progress that soon.
TBH I would also like to fix a known small bug first, though (see below)

Coverage of check,py is %80

Yes indeed ! I have been doing occasional coverage checks, and I think all the actual checking code is well covered, but other less essential aspects are still a bit short of testing.
Notably, since the version 0.1 I find I missed something quite important in the 'summary' output pp-mo/ugrid-checks#20 -- that feature was a late addition, which I did not add tests for, so that just goes to show. I hope to fix it quite soon.

Do you feel that a coverage check should be built into CI testing ?
I'm never sure how I feel about that myself. It's hard to set pass criteria and can sometimes seem like a bit of a pointless box-ticking exercise.

I think we can simply say that 3D support is tentative and not CF compliant.

OK that is encouraging. And I think it makes great sense, in the spirit of your

something mostly done is better than nothing :-)

Hi @ChrisBarker-NOAA quick update.
I have now found time to fix the bugs I identified with 0.1, improve testing, and push that out as version 0.1.1

The coverage is now much higher, and for sure it was worth writing proper tests because I found at least 2 more bugs with the structure-report : ( It really was a bit last-minute and not ready )

What may interest you more ... with a little help from @bjlittle it is now also available on conda-forge 🔔 ✔️
https://github.com/conda-forge/ugrid-checks-feedstock

However ... I have also in this process learnt that there is some prior art I was not previously aware of,
in https://github.com/ioos/cc-plugin-ugrid

So, I will next be engaging with that project and seeing where this takes us.
the connection was made by @ocefpaf in this discussion

I likewise been asked by a couple of colleagues whether this would eventually be subsumed in cf-checker.
I must say, I still don't really have much of a view on that. It seems practically some way off possibility at present, which is why I decided not to start there. Do you have any views on that ?

Latest 390ec65 to address the problem identified pp-mo/ugrid-checks#34

Unfortunately, this demonstrates the lack of oversight on my proposals !

It's also a sufficiently prominent bug that I intend to issue a further v0.1.2 bugfix release for ugrid-checks.

Hi @ChrisBarker-NOAA , did you see this comment, and how it relates to this ?

I'd be happy to work on anything that needs fixing, here, if it means we can establish that link with CF.
That includes adding the 3d mesh implementation (i.e. with volumes), which I avoided doing, if that is the best way of achieving consistency with the main description,
though I guess your earlier suggestion of "you could put a big "DRAFT" note at the top, and then we could merge it" woudl be a quicker win, if that would satisfy needs for now.

I do wonder if this addition really prompts a version increment, though, or even a release cut.
In practice, what shows on ReadTheDocs is a "latest cut", but it shifts further away from the stated "v1.0 release" every time a change is merged.
So, in terms of governance, I think we could be generally more careful about versioning : While there is an official "v1.0", i.e. a release tagged and referenced on GitHub, the visible readthedocs page doesn't currently give you access to that.
That is a pretty easy thing to fix, since RTD supports those concepts of a "latest" , "stable" and "previous" versions very well.
However, I it needs to be managed through the main RTD project page by a maintainer -- i.e. a general contributor like myself can't propose changes, as I would for the repo itself.

As for versioning -- the convention version is a version of the convention itself, not the document. So any changes/updated to the document(s) shouldn't change the version number of the convention.

That being said, it there's something to be said for tracking the version of the documents -- maybe a separate version number, or use the patch version? (e.g. 1.0.3 for a third update to the docs that don't change the convention)

@hrajagers, @rsignell-usgs, @ocefpaf, or anyone else:

I think we should merge this -- a draft is better than nothing, particularly given that UGRID may be officially mentioned in CF soon (cf-convention/cf-conventions#153)

But I'd like at least one other voice of consent :-)

Hello,

UGRID is all set to be incorporated in to CF-1.11, with the only barrier being the creation of the UGRID conformance document. @ChrisBarker-NOAA suggested that merging this PR to publish the conformance rules as a draft would be fine for this purpose, and I agree.

Is this something that you're all happy to do, do you think?

Thanks,
David

Is this something that you're all happy to do, do you think?

For reference, it does contain the slightly radical suggestion to omit the full 3D aspect.
On the other hand, IMHO it can be accepted "as draft", as already suggested.

@pp-mo wrote:

"For reference, it does contain the slightly radical suggestion to omit the full 3D aspect."

As far as I could see -- it's only omitting 3D from the conformance rules -- which doesn't seem radical to me. Or does that mean a 3D mesh would be non-conforming by these rules, as opposed to conformance-unknown.

If it would b enon-conforming, then we could call these conformance rules for < 3D meshes, and be done for now.

Hi - remembering a comment from @hrajagers, 3D meshes are currently not fully defined, because the ordering of the nodes is ambiguous, and the suggestion was to fix the numbering, or drop 3D meshes until they're needed. The latter approach gained some support ... so not including them in the conformance doc seems reasonable to me (although I have no strong opinion, nor sway, here!)