add table for versions numbers - #2

Hi, thanks for this. I am in the process of writing a draft spec document (quite simple) to describe the format, and I came to the same conclusion that we should include a table for sqlitestore-related metadata. I will open a PR soon to serve as an RFC for the spec.

@auxym Great -- yeah, I think versioning would help allow multiple schema. Switching to somehing as complex as "Main is a Table of Groups", and "each Group is Tables of Arrays (and attributes)" and "Each Array as a table of chunks and attributes" would then be an implementation detail, and the code could probably support multiple schema. Support 'em all and see which one is the performance win for each use case.

I will say that I found the discussion over at Zarr-python to be worrying. I'm sure they mean well, but suggesting using S3 when asking about "single file" seems quite strange to me. And the "zarr uses key-value and that maps to Directory/Files" might be even weirder. Actually, key-value maps to Tables trivially. Yes, it could can be mapped to Directory/Files, but that is less obvious, and sort of requires convention/hidden assumptions which, as far as I can tell, are not versioned ;). I think the Zarr developers must be thinking about solutions to different problems.

I'm happy to try to help here, and am hopeful that this can be a great solution.

Adressed in #4

@auxym. Yeah, I guess this PR did not match the Spec.

OTOH, the current implementation doesn't follow the Spec because it does not have a metadata table. This PR attempted that, and it could probably have been fixed to match the Spec, mostly by renaming things. Closing this, you
sort of go back to not meeting the spec and having rejected all attempts to meet the spec.

Is there a plan to implement the metatable table? A plan to release this?

I have no idea if the Spec here gets turned into a "ZEP" -- that process seems completely opaque to me.
I have found the process and interactions with Zarr developers to be extremely discouraging.

Yes sorry, the dev branch currently meets the spec. I had planned to merge it immediately but there's a small issue I just need to fix first. Should be done soon.

@auxym Oh. It looks like you have several active branches. Should it be obvious what the intention is for these different branches or for this code in general? Are there plans to make a release to PyPI?

You had an early outside Pull Request and a chance to double the number of developers, and you closed that PR months later without discussion, and with a short message that was kind of not correct, or at least not the full story.

It is very hard to guess intent in situations like this, but it seems like you do not really want my help.

Wishing you all the best, and hoping to be able to use this code in the near future.

Yes, I had a bunch of branches with experimental code I was working on as the spec was being revised.

Spec-compliant code is now merged in main and the experimental branches will be cleaned up.