-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
System information
- TensorFlow.js version (you are using): 3.11.0
- Are you willing to contribute it (Yes/No): Yes, willing to help where possible
Describe the feature and the current behavior/state.
I've been playing with ONNX Runtime Web recently, and one excellent feature is the ability to move all processing off the main thread by setting ort.env.wasm.proxy = true. This proxies all InferenceSession calls via a web worker when using the wasm backend.
It was a pleasant surprise because getting things to work in a web worker and writing all the glue code can be an unpleasant experience.
It would be great if tfjs had a feature like this.
Will this change the current api? How?
Yes, there would be an API addition to specify that all heavy processing should be done in workers - either globally or per-model, or both.
Who will benefit with this feature?
Processing should almost never be done on the main thread, since it degrades the user experience with UI lag. By doing the work to implement this once, I think many thousands of developers would be saved from individually writing the same worker-glue code by themselves.