Skip to content

Commit b13b7d3

Browse files
committed
dispose session outside of child proc
disposing inside the child proc can interfere with the parent process. we likely never considered this. Fixes: #7815 Change-Id: I6ad0e5840655ed99a9d30002eba280c8e44a5c2e (cherry picked from commit 936f0b2)
1 parent ecc2bbf commit b13b7d3

1 file changed

Lines changed: 30 additions & 11 deletions

File tree

‎doc/build/core/pooling.rst‎

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -479,19 +479,34 @@ are three general approaches to this:
479479
engine = create_engine("mysql://user:pass@host/dbname", poolclass=NullPool)
480480

481481

482-
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine` as
483-
soon one is within the new process. In Python multiprocessing, constructs
484-
such as ``multiprocessing.Pool`` include "initializer" hooks which are a
485-
place that this can be performed; otherwise at the top of where
486-
``os.fork()`` or where the ``Process`` object begins the child fork, a
487-
single call to :meth:`_engine.Engine.dispose` will ensure any remaining
488-
connections are flushed. **This is the recommended approach**::
482+
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine`
483+
**directly before** the new process is started, so that the new process
484+
will create new connections, as well as not attempt to close connections that
485+
were shared from the parent which can impact the parent's subsequent
486+
use of those connections. **This is the recommended approach**::
489487

490488
engine = create_engine("mysql://user:pass@host/dbname")
491489

492490
def run_in_process():
493-
# process starts. ensure engine.dispose() is called just once
494-
# at the beginning
491+
with engine.connect() as conn:
492+
conn.execute(text("..."))
493+
494+
# before process starts, ensure engine.dispose() is called
495+
engine.dispose()
496+
p = Process(target=run_in_process)
497+
p.start()
498+
499+
3. Alternatively, if the :class:`_engine.Engine` is only to be used in
500+
child processes, and will not be used from the parent process subsequent
501+
to the creation of child forks, the dispose may be within the child process
502+
right as it begins::
503+
504+
engine = create_engine("mysql+mysqldb://user:pass@host/dbname")
505+
506+
def run_in_process():
507+
# process starts. ensure engine.dispose() is called just once
508+
# at the beginning. note this cause parent process connections
509+
# to be closed for most drivers
495510
engine.dispose()
496511

497512
with engine.connect() as conn:
@@ -500,10 +515,14 @@ are three general approaches to this:
500515
p = Process(target=run_in_process)
501516
p.start()
502517

503-
3. An event handler can be applied to the connection pool that tests for
518+
# after child process starts, "engine" above should not be used within
519+
# the parent process for connectivity, without calling
520+
# engine.dispose() first
521+
522+
4. An event handler can be applied to the connection pool that tests for
504523
connections being shared across process boundaries, and invalidates them.
505524
This approach, **when combined with an explicit call to dispose() as
506-
mentioned above**, should cover all cases::
525+
mentioned above in options 2 or 3**, should cover all cases::
507526

508527
from sqlalchemy import event
509528
from sqlalchemy import exc

0 commit comments

Comments
 (0)