Skip to content

Jupyter + ROOT: too many open files #8326

@PetrilloAtWork

Description

@PetrilloAtWork
  • Checked for duplicates

Describe the bug

On Linux, in a Juyter/ROOT Python notebook, every cell execution makes the python process open two "files":

lrwx------ 1 user users 64  2 Jun 22.33 77 -> /dev/pts/7
lr-x------ 1 user users 64  2 Jun 22.33 78 -> 'pipe:[214404]'

one to the terminal bound to the notebook server (/dev/pts/7), and another always a different pipe reading for nothing to write in it, e.g. from lsof:

python3.8 16586                           user   78r     FIFO               0,10       0t0     214404 pipe

These stay open, making the process reach the OS limit of open files sooner or later, at which point grief ensues.
Some commands (e.g. matplotlib figures) may create additional pairs out of the void.

Expected behavior

The open file descriptor count should be roughly stable and not proportional to the number of executed cells.

To Reproduce

jupyter --notebook # or root --notebook

then execute (empty lines mark cell boundaries):

import os
def printFiles(): print(f"PID={os.getpid()} files={len(os.listdir(f'/proc/{os.getpid()}/fd'))}")
printFiles()

printFiles()

printFiles()

printFiles()

import ROOT
printFiles()

printFiles()

printFiles()

printFiles()

I get a file count of 43 or 44 on each cell before importing ROOT, and from the next cell, each cell increments by 4 (48, 52, ...).
The issue does not appear on plain python/ipython.

Setup

Working with:

  • Gentoo Linux, amd64, kernel 5.12.4 (last update a few days ago)
  • ROOT 6.24/00
  • Python 3.8.10 (dev-lang/python-3.8.10_p1 from Gentoo)
  • Jupyter 1.0.0
  • ipython 7.24.0

I assume this issue is setup-dependent, or it would have been reported already... (actually, it was in 2017).

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions