Recently I wrote some code to test the performance of Scala benchmark Inference and found that it shows almost equal performance with single inference and batch inference. The core problems are the rebind on the input if you feed in a batch of images. This should be addressed to release the full strength of Scala batch inference.