Skip to content

Commit c7c6b70

Browse files
committed
dispose session outside of child proc
disposing inside the child proc can interfere with the parent process. we likely never considered this. Fixes: #7815 Change-Id: I6ad0e5840655ed99a9d30002eba280c8e44a5c2e (cherry picked from commit 936f0b2)
1 parent fc7f889 commit c7c6b70

1 file changed

Lines changed: 30 additions & 11 deletions

File tree

‎doc/build/core/pooling.rst‎

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -426,19 +426,34 @@ are three general approaches to this:
426426
engine = create_engine("mysql://user:pass@host/dbname", poolclass=NullPool)
427427

428428

429-
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine` as
430-
soon one is within the new process. In Python multiprocessing, constructs
431-
such as ``multiprocessing.Pool`` include "initializer" hooks which are a
432-
place that this can be performed; otherwise at the top of where
433-
``os.fork()`` or where the ``Process`` object begins the child fork, a
434-
single call to :meth:`_engine.Engine.dispose` will ensure any remaining
435-
connections are flushed. **This is the recommended approach**::
429+
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine`
430+
**directly before** the new process is started, so that the new process
431+
will create new connections, as well as not attempt to close connections that
432+
were shared from the parent which can impact the parent's subsequent
433+
use of those connections. **This is the recommended approach**::
436434

437435
engine = create_engine("mysql://user:pass@host/dbname")
438436

439437
def run_in_process():
440-
# process starts. ensure engine.dispose() is called just once
441-
# at the beginning
438+
with engine.connect() as conn:
439+
conn.execute(text("..."))
440+
441+
# before process starts, ensure engine.dispose() is called
442+
engine.dispose()
443+
p = Process(target=run_in_process)
444+
p.start()
445+
446+
3. Alternatively, if the :class:`_engine.Engine` is only to be used in
447+
child processes, and will not be used from the parent process subsequent
448+
to the creation of child forks, the dispose may be within the child process
449+
right as it begins::
450+
451+
engine = create_engine("mysql+mysqldb://user:pass@host/dbname")
452+
453+
def run_in_process():
454+
# process starts. ensure engine.dispose() is called just once
455+
# at the beginning. note this cause parent process connections
456+
# to be closed for most drivers
442457
engine.dispose()
443458

444459
with engine.connect() as conn:
@@ -447,10 +462,14 @@ are three general approaches to this:
447462
p = Process(target=run_in_process)
448463
p.start()
449464

450-
3. An event handler can be applied to the connection pool that tests for
465+
# after child process starts, "engine" above should not be used within
466+
# the parent process for connectivity, without calling
467+
# engine.dispose() first
468+
469+
4. An event handler can be applied to the connection pool that tests for
451470
connections being shared across process boundaries, and invalidates them.
452471
This approach, **when combined with an explicit call to dispose() as
453-
mentioned above**, should cover all cases::
472+
mentioned above in options 2 or 3**, should cover all cases::
454473

455474
from sqlalchemy import event
456475
from sqlalchemy import exc

0 commit comments

Comments
 (0)