In Chromium CL review, @fdwr mentioned (Thanks Dwayne!)

0 size dimensions really should just be treated as nops. e.g. memcpy(dest, src, 0) is valid and does nothing, and adding two empty tensors just returns an empty tensor. There are legitimate cases within a graph where a tensor may be temporarily sliced down to emptiness and then reconcatenated later with other data (I've come across at least one ONNX model that does this).

From native ML API perspective, Dwayne also mentioned

Now, there are certain backends that are not prepared for empty dimensions (e.g. DML doesn't accept them and rejects the operator creation call), and those operators will need to be skipped inside the backend, but the front-end model builder imo should treat them validly and return 0 elements.

We may want to investigate more native ML APIs to understand what's the status of the support.

I am opening this issue to start tracking, e.g. adding TODO in the implementation. @fdwr, feel free to share more details. Thanks!