-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Jupyter + ROOT: too many open files #8326
Description
- Checked for duplicates
Describe the bug
On Linux, in a Juyter/ROOT Python notebook, every cell execution makes the python process open two "files":
lrwx------ 1 user users 64 2 Jun 22.33 77 -> /dev/pts/7
lr-x------ 1 user users 64 2 Jun 22.33 78 -> 'pipe:[214404]'
one to the terminal bound to the notebook server (/dev/pts/7), and another always a different pipe reading for nothing to write in it, e.g. from lsof:
python3.8 16586 user 78r FIFO 0,10 0t0 214404 pipe
These stay open, making the process reach the OS limit of open files sooner or later, at which point grief ensues.
Some commands (e.g. matplotlib figures) may create additional pairs out of the void.
Expected behavior
The open file descriptor count should be roughly stable and not proportional to the number of executed cells.
To Reproduce
jupyter --notebook # or root --notebook
then execute (empty lines mark cell boundaries):
import os
def printFiles(): print(f"PID={os.getpid()} files={len(os.listdir(f'/proc/{os.getpid()}/fd'))}")
printFiles()
printFiles()
printFiles()
printFiles()
import ROOT
printFiles()
printFiles()
printFiles()
printFiles()I get a file count of 43 or 44 on each cell before importing ROOT, and from the next cell, each cell increments by 4 (48, 52, ...).
The issue does not appear on plain python/ipython.
Setup
Working with:
- Gentoo Linux, amd64, kernel 5.12.4 (last update a few days ago)
- ROOT 6.24/00
- Python 3.8.10 (
dev-lang/python-3.8.10_p1from Gentoo) - Jupyter 1.0.0
- ipython 7.24.0
I assume this issue is setup-dependent, or it would have been reported already... (actually, it was in 2017).