The two set_num_interop_threads, get_num_interop_threads CPU threading variables explained here for PyTroch and for TensorFlow have a huge impact on CPU inferencing time, e.g. for Resnet 18 TorchVistion image model under one CPU core assignment it results in the following difference in latencies (before applying and after applying these values). I think it is worthwhile adding this two variable as a configurable setting variable at least for Huggingface runtime that is using deep models (and I can validate I have seen the same trends for many Huggingface pipeline models too).