Skip to content

API simplification: context owns builder, graph becomes internal slot #303

@zolkis

Description

@zolkis

Lifted from #298 comment.

Proposal: include graph builder, command encoder as attributes to MLContext, and make MLGraph an internal slot of MLContext

This would simplify things a great deal and would make things more consistent: avoid exposing the empty MLGraph interface in the spec, and using it as internal slot would allow differentiation of how it can be used in different context types and also manage its lifecycle.

[SecureContext, Exposed=(Window, DedicatedWorker)]
interface MLContext {
  // graph is an internal slot (eventually in different forms during lifecycle) 
  // lifecycle state could also be an internal slot, e.g. 
  //    "building", "compiled", "encoding", "encoded", "compute" etc.

  readonly attribute MLGraphBuilder builder;  // dedicated builder operates on internal slots
  readonly attribute MLCommandEncoder? commandEncoder;  // only defined for "webgpu" context

  Promise<MLNamedArrayBufferViews> compute(MLNamedArrayBufferViews inputs);  
};

Rationale for change

The spec contains certain constraints that are hard to describe and enforce via algorithms. For instance, the note from the [MLContext section(https://webmachinelearning.github.io/webnn/#api-mlcontext):

When the [[contextType]] is set to default with the MLContextOptions.deviceType set to gpu, the user agent is responsible for creating an internal GPU device that operates within the context and is capable of ML workload submission on behalf of the calling application. In this setting however, only ArrayBufferView inputs and outputs are allowed in and out of the graph execution since the application has no way to know what type of internal GPU device is being created on their behalf. In this case, the user agent is responsible for automatic uploads and downloads of the inputs and outputs to and from the GPU memory using this said internal device.

Or, from the MLCommandEncoder section,

Record the initialization of the MLGraph. This is a necessary step for optimal performance during graph execution as it gives the platform an opportunity to prepare and optimize constant input data for the subsequent execution of the graph. This method should only be called once per graph.

To achieve that, there should be a possibility of binding MLGraph and MLCommandEncoder to an MLContext of type "webgpu".

Therefore I would add an internal slot [[model]] to MLContext that represents a compute graph bound to a context. If that context is of type "webgpu", then it will have MLCommandEncoder-specific initialization, dispatch and finish (e.g. the MLCommandEncoder interface could be exposed in the context as an attribute).

Also, the discussion in #149 reveals a possible use case of discerning between a compute graph (as built by a builder, that could be executed in a multiple contexts) and a graph that is initialized for a given context for execution.
A builder could be generic (initialized without a context) or bound to a context (when adaptation to the context could already happen during the build).
A context-bound builder's build() produces MLGraph that is already bound to that context.
A generic builder's build() could have a parameter for context for which the MLGraph is built.

In summary: a builder's output, i.e. an MLGraph is always bound to an MLContext, so it could as well be (part of ) an internal slot of MLContext, and also the builder could be an attribute of MLContext.
As noted previously, MLCommandEncoder could also be an attribute of MLContext.

Related to #302.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions