Not sure about the cost of this vs. using function_utils::CreateSchema or something similar to dynamically create a schema as needed.

It makes more sense to me if we also have a CPU EP implementation of the operator.

It makes less sense to me if it's XNNPACK specific, as you're going to have to create/maintain a schema for every quantized operator (including multiple opsets) that we add XNNPACK support for.

What value does this static schema provide that a dynamically created schema does not?

A CPU implementation is here. #12177
I will refactor it after that PR is finished.

OK.

Softmax isn't a layout sensitive operator though so the domain should not be onnxruntime::kMSInternalNHWCDomain. We only want to use that for operators that specifically take NCHW input.

The transpose optimizer has a list of layout sensitive ops if you ever need to check.

Some extra info. When the layout transformer runs (TransformLayoutForEP) it will replace each layout sensitive node with a node in the NHWC domain, and wrap Transpose operators around that node to convert from the original NCHW to NHWC. That limits the new nodes in the kMSInternalNHWCDomain domain to this list, along with a few ORT specific ones listed here.

Once that process finishes, the transpose optimizer will move the Transpose nodes and cancel as many out as possible, but that process does not involve changing the domain of the other nodes. Due to this the Softmax should stay in the ONNX domain.

Say we the EP asked for Conv -> Conv -> Softmax. First step would create Transpose(to NHWC) -> Conv(kMSInternalNHWCDomain) -> Transpose(to NCHW) -> Transpose(to NHWC) -> Conv(kMSInternalNHWCDomain) -> Transpose(to NCHW) -> Softmax.

Second step would cancel out most of the Transpose ops, and push the last one past the Softmax so it's at the boundary of the nodes requested by the EP to maximize the nodes the EP can run in NHWC layout. That results in Transpose(to NHWC) -> Conv(kMSInternalNHWCDomain) -> Conv(kMSInternalNHWCDomain) -> Softmax -> Transpose(to NCHW).

Thanks for explaning.
On top of that, It should be fine to register only Layout-sensitive Ops in kMSInternalNHWCDomain, and OutputShape inferencing is the same as onnx Ops.

Sorry - still not understanding this. A NodeUnit has QuantParam info in the input/output defs so why do we need to duplicate it here instead of using NodeUnit?

Also not understanding 'all ops will have at least X or W or Y'. Softmax only has one input. MatMul has 'A' and 'B' not and 'X' and 'W'. The quant param info in NodeUnit is attached to each input/output and will match exactly what the node uses.

OK.

Softmax isn't a layout sensitive operator though so the domain should not be onnxruntime::kMSInternalNHWCDomain. We only want to use that for operators that specifically take NCHW input.

The transpose optimizer has a list of layout sensitive ops if you ever need to check.

We can do it in a separate PR.

I think we need to try and support both the input and activation nodes being be a QDQ group. We should think about things in terms of NodeUnit and whether we can fuse one NodeUnit to another. A supported NodeUnit will have an entry in the map to the ComputeCapability. When fusing we're updating that ComputeCapability. It's just that there may be more steps if they're QDQ groups (e.g. use the zp/scale from the activation node output in the ComputeCapability).