-
Notifications
You must be signed in to change notification settings - Fork 38
Description
A fully-loaded current v2 message format looks like this when represented as semi-pretty-printed dag-json (data taken from the roundtrip test, this is not representative of a real message, it contains everything)):
{
"Blks": [
{
"Data": {"/":{"bytes":"QgTLmh40xfCOmyCqdgkOcAILtWwMo9OvcpbNEFilESiQ/tIYSI8ITY355INftUrQRf/ZNuO/cmGwQmxRNSoJeBbtdEgruQhLSn7YrcUX8zceDgQ0tRFiXNGkF5IkPczc/ogJSw"}},
"Pre": {"/":{"bytes":"AVUSIA"}}
},
{
"Data": {"/":{"bytes":"xfPTKlWZ2kO4US4NsU11HtHWQJ2dy3rtBdbMbAmHxjTufEI28FF8INzGqZ6G5LrcAl5fXoG0WWEoUimfNvEQ6Qa6IW4ByDcDUq0ziUrrH+WNzNCC9FaSKBovz42VowZ4zaRVhw"}},
"Pre": {"/":{"bytes":"AVUSIA"}}
}
],
"Reqs": [
{
"Canc": false,
"Ext": {
"AppleSauce/McGee": "yee haw"
},
"ID": {"/":{"bytes":"v8HW/fqRRd65WDICpHzhIw"}},
"Pri": 101,
"Root": {"/":"bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi"},
"Sel": {"R":{":>":{"|":[{".":{}},{"a":{">":{"@":{}}}}]},"l":{"none":{}}}},
"Updt": false
},
{
"Canc": false,
"Ext": {},
"ID": {"/":{"bytes":"wSjEHwARQ66IR7wP7nOl0g"}},
"Pri": 202,
"Root": {"/":"bafyreibdoxfay27gf4ye3t5a7aa5h4z2azw7hhhz36qrbf5qleldj76qfy"},
"Sel": {"R":{":>":{"a":{">":{"@":{}}}},"l":{"none":{}}}},
"Updt": false
}
],
"Rsps": [
{
"Ext": {},
"ID": {"/":{"bytes":"v8HW/fqRRd65WDICpHzhIw"}},
"Meta": [
[
{"/":"bafyreibdoxfay27gf4ye3t5a7aa5h4z2azw7hhhz36qrbf5qleldj76qfy"},
"m"
]
],
"Stat": 34
},
{
"Ext": {
"Hippity+Hoppity": {"/":{"bytes":"9V/48SUItj7yv+ynVXrpDfYxGl7BYxtKH6hDMQvZw6cQ6qzlob3XKtC/4El3HBHnVjOL2Thl5kXxreybnJnvQH+9T8aFnnkExa19yb0QpcwWlz1bKOwabdQ9n4L58Yw9A0GONQ"}}
},
"ID": {"/":{"bytes":"wSjEHwARQ66IR7wP7nOl0g"}},
"Meta": [],
"Stat": 14
}
]
}Current IPLD schema is https://github.com/ipfs/go-graphsync/blob/83c3860142cfbed18b8d87a6b3d375d017ef4fc8/message/ipldbind/schema.ipldsch
There's two primary things that I'd like to address here in a follow-up PR with a more concrete proposal:
- Size: the current format is a bit excessive with the string keys, I'd like to try and minimise this a bit more, mainly by leaning on tuple representation for most of the important bits.
- Additional versioning options: we have protocol version switching at the libp2p layer, but we could also be providing ourselves some optionality at the IPLD layer (https://ipld.io/docs/schemas/using/migrations/ is a good reference for this).
On this second item—by wrapping in a top-level single key keyed union we can iterate on this protocol without being forced to do it only at the libp2p layer. Wrapping in something like {"gs2":{....}} lets us do this. This way we can even iterate within the single, main schema, just by adding another key to the union. We get to reuse any of the overlapping types and make breaking adjustments without actually breaking the protocol. Implementations (this and Myel's one and anyone else's implementation) would just need to be aware that a version signal exists at this top-level union as well so any "can't decode according to schema" could optionally be translated as "don't know how to deal with protocol v2 / gsX" or some such.
So we could evolve the schema in 3 ways over time:
- Backward compatible within the current schema, which for tuples would mostly mean adding new items to the end and inserting null entries where things go missing.
- Breaking the schema and adding a new key to the top-level union, which gives us a lot of flexibility to move things around - perhaps we've extended the tuples a whole lot and nullified existing entries and they're getting out of control so we reset by doing this.
- Adding a new libp2p protocol version and doing an even harder break.
Having the optionality and doing some additional descriptive in-protocol signalling are the main goals though.