Hi @PhilipMay, I added a suggestion commit with a call to the close_multiprocessing_pool() each time we use the Inferencer. In that case, we might not need the warning comments. What do you think?

Hi @PhilipMay, I added a suggestion commit with a call to the close_multiprocessing_pool() each time we use the Inferencer. In that case, we might not need the warning comments. What do you think?

Well, that does not fully work because you would have to wrap the code into a try and finally block to even close the pool when an exception is raised.

And to avoid this kind of "ugly and complicated" code I would better not open a pool... So the example is easy to read.

Hi @PhilipMay,

In the case of exception in the Inferencer, the example scripts would crash, terminating all the Python subprocesses. For instance, the following script should exit clean without any zombie processes.

results = inferencer.inference_from_dicts(qa_input)
print(results)
raise Exception  # scripts exits here
inferencer.close_multiprocessing_pool()

@tanaysoni Yes I agree. In the example it does not cause any direct problems. But IMO an example should be something that can be taken and put into something bigger. If placed in a real life server environment this might cause an ugly memory leak if not handled correctly. But we already had this discussion on an other PR if I remember correctly.

So what do you suggest we can do with this uncompleted PR? In which direction should we go?

Hi @PhilipMay, IMO we add a call to close_multiprocessing_pool() in each example along with a single line comment mentioning it's important to close the pool. Additionally, we can write a detailed example explaining more details about the Pool behavior. What do you think?

Hi @PhilipMay, thanks again for bringing this up and working on it! I added an additional example inferencer_multiproccesing.py and rebased the branch with master. If you think there's more documentation we could add, let's discuss and take them up in a separate PR.

Ok. Thanks. 👍