Repository navigation
Key encoding for dbm-style databases #133
Description
There are a number of embedded key-value stores available, including the built-in dbm-style databases provided by the Python standard library, as well as BerkeleyDB and Kyoto Cabinet. These provide a MutableMapping interface but generally expect keys to be bytes objects not strings, and will return bytes objects when asked for an iterator over keys. Zarr uses text strings for keys currently, and so these key-value databases cannot be used directly with Zarr, however it would be nice and should be straighforward to provide some support.
There are two options. (1) Provide a MutableMapping wrapper class that performs key encoding/decoding (e.g., 'zarr.storage.KeyValueStore' or 'zarr.storage.DbmStore'). (2) Add a key_encoding option to the Array and Group classes, which would be None by default (preserving current behaviour to use text strings as keys) but if not None would be used to encode and decode keys outside of the store.
Activity
[-]Key encoding[/-][+]Key encoding for dbm-style databases[/+]alimanfoo commentedon Feb 27, 2017
I'm leaning towards (2) as it seems reasonable and avoids yet another layer of indirection for the user.
jakirkham commentedon Feb 27, 2017
One thing to consider is whether this will open the door to handling a variety of different encodings.
Also related, will the encoding information be stored somehow?
alimanfoo commentedon Feb 27, 2017