We need to define and document behavior of Any, a named type that can be used in IPLD Schemas to say, well, "anything" goes here. This is both a documentation and a design issue.

As #301 comments, Any is something we can implement in a very non-magical way, simply by specifying it in the prelude as a kinded union:

type Any union {
	| Bool   bool
	| Int    int
	| Float  float
	| String string
	| Bytes  bytes
	| Map    map
	| List   list
	| Link   link
} representation kinded

Doing this as a bog-standard kinded union has very pleasing parsimony.

However, it has some interesting implications. Namely, since it's just a regular kinded union in the IPLD Schema system, it's got the dichotomy of type-level and representation-level semantics. And generally, what users want and expect of this is the representation-level semantics. But they get... more.

So the question is essentially: is this definition of Any actually good?

The general rule of how unions work is: the logical-level always behaves as a map, where the keys are the names of the types. The representation-level, of course, varies wildly (e.g. kinded vs keyed unions behave nothing alike, representationally).

So, if Any is implemented as a kinded union, it does exactly that: the representation-level behaviors are "put anything here", as desired; and the type-level semantics... well, it acts like a map: sometimes the map will have an entry with the key "String", sometimes it'll have an entry instead with the key "Map", etcetera.

There are some POVs from which I don't think it violates the principle of least surprise. But, there are definitely also POVs from which it does. Namely, a typed-Any doesn't really work the same as an untyped-basicnode.Any.

So maybe something not great is going on here with this design. For somegenoutput.Any to behave differently than basicnode.Any is a significant smell.

One option -- Option 0, if you will -- is to roll with this (and that's what the current tip of go-ipld-prime codegen is actually doing. Or, er, well, it's what it recommends doing -- since the prelude is still just a suggestion there, not baked in. Still! Moving on!).

One option for dealing with this is changing the behavior of the type level node for kinded unions. We could declare that for kinded unions, the type-level behavior is actually the same as the representation-level behavior.

I'm a pretty dubious of this: it would break one of the high level design rules of the schema system: that type-level behaviors are defined entirely in terms of the type kind, and have total independence from their representation. (On the other hand, this is admittedly a pretty compelling case.)

Another option is that we make a blessed magic type for Any. We give this magic type the behaviors that we want (essentially, it acts like basicnode.Any: there's no difference between it's type-level and representational-level behaviors, and the representational behavior is identical to the kinded union specified above.) In this case, type Any [...] wouldn't end up in the prelude at all: it would be too magical to describe.

This is quite a bit less parisimonious than just defining Any as a plain, non-magical kinded union. But I really don't have any arguments against it other than that.

The third option is to carry that previous idea a bit further: we could introduce a new kind named any (e.g. it's a conceptual sibling of list|string|map|bytes|etc). We would imbue this kind with the behaviors we want: "put anything here" and "the type level and representation level semantics are the same". The prelude would then also simply contain type Any any (just like it contains type String string).

This seems pretty good. We're not breaking the high-level rule about type/representation independence with this approach, since we just introduce a whole new kind and it just "happens" to have specified the type behaviors and representation (of which there is only one) behaviors as the same.

Probably the weirdest thing about this idea is that ... hmm. No, nothing's weird about this at all. any would be a type-kind, not a representation-kind (e.g., it's in the enum that also contains struct|union|etc), and adding another member to that enum isn't problematic at all. (There was a moment where I thought we'd be stuct introducing a third enum which included this extra weird member, but... no, I think that was sloppy thinking; it's actually simpler than that.)

Okay. I think I talked myself into favoring Option 3 here in the course of writing this issue :) Anyone else have any thoughts on this?

Thanks to @mvdan and @willscott for both independently poking me about something feeling a bit weird here, and thus encouraging this further thought.