Skip to content

Commit 8857417

Browse files
committed
dispose session outside of child proc
disposing inside the child proc can interfere with the parent process. we likely never considered this. Fixes: #7815 Change-Id: I6ad0e5840655ed99a9d30002eba280c8e44a5c2e (cherry picked from commit 936f0b2)
1 parent 57552b6 commit 8857417

1 file changed

Lines changed: 30 additions & 11 deletions

File tree

‎doc/build/core/pooling.rst‎

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -446,19 +446,34 @@ are three general approaches to this:
446446
engine = create_engine("mysql://user:pass@host/dbname", poolclass=NullPool)
447447

448448

449-
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine` as
450-
soon one is within the new process. In Python multiprocessing, constructs
451-
such as ``multiprocessing.Pool`` include "initializer" hooks which are a
452-
place that this can be performed; otherwise at the top of where
453-
``os.fork()`` or where the ``Process`` object begins the child fork, a
454-
single call to :meth:`_engine.Engine.dispose` will ensure any remaining
455-
connections are flushed. **This is the recommended approach**::
449+
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine`
450+
**directly before** the new process is started, so that the new process
451+
will create new connections, as well as not attempt to close connections that
452+
were shared from the parent which can impact the parent's subsequent
453+
use of those connections. **This is the recommended approach**::
456454

457455
engine = create_engine("mysql://user:pass@host/dbname")
458456

459457
def run_in_process():
460-
# process starts. ensure engine.dispose() is called just once
461-
# at the beginning
458+
with engine.connect() as conn:
459+
conn.execute(text("..."))
460+
461+
# before process starts, ensure engine.dispose() is called
462+
engine.dispose()
463+
p = Process(target=run_in_process)
464+
p.start()
465+
466+
3. Alternatively, if the :class:`_engine.Engine` is only to be used in
467+
child processes, and will not be used from the parent process subsequent
468+
to the creation of child forks, the dispose may be within the child process
469+
right as it begins::
470+
471+
engine = create_engine("mysql+mysqldb://user:pass@host/dbname")
472+
473+
def run_in_process():
474+
# process starts. ensure engine.dispose() is called just once
475+
# at the beginning. note this cause parent process connections
476+
# to be closed for most drivers
462477
engine.dispose()
463478

464479
with engine.connect() as conn:
@@ -467,10 +482,14 @@ are three general approaches to this:
467482
p = Process(target=run_in_process)
468483
p.start()
469484

470-
3. An event handler can be applied to the connection pool that tests for
485+
# after child process starts, "engine" above should not be used within
486+
# the parent process for connectivity, without calling
487+
# engine.dispose() first
488+
489+
4. An event handler can be applied to the connection pool that tests for
471490
connections being shared across process boundaries, and invalidates them.
472491
This approach, **when combined with an explicit call to dispose() as
473-
mentioned above**, should cover all cases::
492+
mentioned above in options 2 or 3**, should cover all cases::
474493

475494
from sqlalchemy import event
476495
from sqlalchemy import exc

0 commit comments

Comments
 (0)