MLOperand may not need to associate with an MLActivation. It may need a slot for the operator (e.g., platform object represents the node within the platform graph) that produces this MLOperand.

Yes, I agree this was meant as an internal slot for the operator. But currently MLOperator is MLActivation. I think we might need both. I expressed my doubts in #335

Should |type| be passed? Or just use {{MLOperandDescriptor/type}} instead?

Should the data type of element be validated, to be unsigned long? The value 0 (0 size dimension) should also be invalid.

I don't know, I guess all integer type values should be accepted (the first step checks that), the question is range.

The range should be [1, ULONG_MAX].

I think that may be implementation dependent. What we can explicitly check is that it's a positive ulong, but the implementation may have other constraints that it should be able to tell the API user if the specified dimension is not supported for whatever reason.

Should MLOperand have corresponding platform object? Should the kind, type, dimensions and values be used to create the platform object?

Made a phrasing for that, let's check.

Because the next step is to register this to underlying platform, I am not sure whether MLOperand needs to keep a reference to |bufferView|. Any use cases?

The use case is for that step when registering to the underlying platform IIUC.

I don't think |activation| should be used here. |activation| is only used as activation function that may be fused into an operator. Instead, could the platform object representing an operator be used? The operator produces this MLOperand.

I agree, but it would be nice to have a definition for an operator. An obscure internal object exposed in an internal slot doesn't sound like a clear definition. :)
I wonder if defining MLOperator and also keeping MLActivation would work better? since now we have only MLActivation defined in the spec but MLOperator is used in the implementation.

In that sense, MLOperator would be similarly empty object like MLGraph currently is. Of course Occam tells that if we can do without MLOperator in the spec, we should. But then we need a clear "private" definition of an operator object (for internal use).

Ditto, may need to create the platform object representing this "input" MLOperand.

Because the next step is to register this to underlying platform, I am not sure whether MLOperand needs to keep a reference to |bufferView|. Any use cases?

Should we make a request to "platform graph builder"? Should the "platform graph builder" be an internal slot of MLGraphBuilder |builder|?

For text "as a constant", should we map it to the common computational graph concept?

Let's say MLOperands are actually the "edges" within the graph, while the operators are "nodes". For a node, the input edges represent the input operands of this operator, the output edges represent the output operands of this operator. There could be three types of edge: 1) input edge that only points to a node ("to" node) without the "from" node. It represent the graph input; 2) intermediate edge that connects two nodes, "from" node and "to" node; 3) output edge that only has a "from" a node without a "to" node.

For WebNN MLOperand kind,

• "input" MLOperand should map to input edge of the computational graph.
• "constant" MLOperand should also map to input edge, bound with static data.
• "output" MLOperand is always has the "from" node. It may be two possible types of edge:
  • When it is consumed by another operator, say used as an input operand for an operator build method, builder.relu(x), it turns to an intermediate edge.
  • When it is passed to build(MLNamedOperands outputs) method in the outputs argument, it turns to the output edge of the computational graph.

It might be helpful to refer to a platform graph API, such as DirectML DML_GRAPH_DESC.

Besides "register", I suppose the MLOperand should keep the platform operand object (or the edge of platform computational graph) created by platform graph builder as an internal slot. This platform object should be passed to the platform graph builder when creating an operator (or the node of the platform computational graph).

Thanks Ningxin, looks like this uncovered a lot of things :).
I can do all these, but it is hard to formally use here a graph concept that is defined later (by implementations).

I agree we should define the elements of the structure in the spec, by internal slots (the edge position you described above) and also reference examples like DirectML as you suggested.

We can/should define algorithms for what it means to "register as a constant" or as an output, associating an internal slot for them.

These "create MLOperand" and "create MLActivation" are anyway meta-algorithms, used internally in the spec, so anything build by them is a temporary step, will be completed when these are referenced from the methods of MLGraphBuilder. So it is there many of the things described in your later comments would happen.

All in all, I think we need a revamp of the programming model, MLGraphBuilder etc. Should I do that in this PR?

All in all, I think we need a revamp of the programming model, MLGraphBuilder etc. Should I do that in this PR?

@zolkis yes, the smaller the PRs the easier to review and discuss. I'm not sure if stacked PRs work here but that could be one solution.

I have experimented how these slots would play out.

I ended up making minimal changes for this, removing the unneeded slots, but did not introduce new slots for platform objects. Will do that later if needed - at the moment they are not needed and can be referred to in prose.

An "output" operand should be created from one of the operator build methods. I think the "platform operator" (or the node of the platform computational graph) that produces this operand should be passed as the next argument and it should be kept in the [[operator]] internal slot.

I am not familiar with [=function=]. Does it mean a JavaScript function? Would operator be the underlying platform operator object (or the node of the platform computational graph)?

I think I meant the latter and this needs some rework. For denoting a platform object ('s reference), the internal slot will be a generic object (that can be a function as well).

Is [[tensorData]] needed to be an internal slot? I suppose the values of ArrayBufferViews should be passed to platform graph builder when "register" the constant. What's the use case of it?

Well, that is the use case for it, AFAICT. The "create MLOperand" steps are continued in the builder methods, but they might need the slot for tensor data. But if something is not needed (no use case), will be removed.

The range should be [1, ULONG_MAX].

Should the type MLActivationOptions be an union of all supported options type, say

typedef (MLClampOptions or MLLeakyReluOptions or ...) MLActivationOptions;

Yes, it's the union of all supported options.
Should we explicitly enumerate all of them in a typedef (that would cover a gap in the Web IDL), or should we use a generic dictionary and specify in prose the current meaning? (that would not change the Web IDL, but would clarify the prose).
I do see value in both approaches.
@anssiko do you have a preference?

@zolkis in general if we can define something in Web IDL we should prefer that way over prose. Web IDL is machine-readable and allows for bindings code generation and testing.

SGTM. It's a bit unusual to have such long unions, currently 12 of them, but we can easily revert any time, in the case an external reviewer finds that awkward.

Filed #363 to track this, if we want to.
For now I will go with the explicit union then.

I don't think you can have a WebIDL union of dictionaries, since dictionaries aren't distinguishable and union members must be distinguishable.

Thanks Dom for that reminder. Looks like we are going with option 1, i.e. generic dictionary + specific spec prose in algorithms.

Why do we need to define a new type called MLActivationOptions? An MLActivation is already an abstract interface that can sufficiently represent all available activation types (that are available for operator fusions) e.g. the clamp activation as an MLActivation interface is already created wtih the specific MLClampOptions accordingly.

Also a union of all relevant activation options is unnecessary since the use case for each will only always require just one concrete activation option type e.g. MLClampOptions for clamp activation, etc.

Chai, are you saying that instead of a union of MLActivationOptions we should define a separate type for each respective activation option?

I sympathize with that, and removing the generic object typedef - it doesn't add much value indeed. It is like specifying function interface with generic object parameters.

Removed MLActivationOptions.

Why do we need to define a new type called MLActivationOptions? An MLActivation is already an abstract interface that can sufficiently represent all available activation types (that are available for operator fusions) e.g. the clamp activation as an MLActivation interface is already created wtih the specific MLClampOptions accordingly.

Also a union of all relevant activation options is unnecessary since the use case for each will only always require just one concrete activation option type e.g. MLClampOptions for clamp activation, etc.