Skip to content

Commit 936f0b2

Browse files
committed
dispose session outside of child proc
disposing inside the child proc can interfere with the parent process. we likely never considered this. Fixes: #7815 Change-Id: I6ad0e5840655ed99a9d30002eba280c8e44a5c2e
1 parent d866eec commit 936f0b2

1 file changed

Lines changed: 30 additions & 11 deletions

File tree

‎doc/build/core/pooling.rst‎

Lines changed: 30 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -461,19 +461,34 @@ are three general approaches to this:
461461
engine = create_engine("mysql+mysqldb://user:pass@host/dbname", poolclass=NullPool)
462462

463463

464-
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine` as
465-
soon one is within the new process. In Python multiprocessing, constructs
466-
such as ``multiprocessing.Pool`` include "initializer" hooks which are a
467-
place that this can be performed; otherwise at the top of where
468-
``os.fork()`` or where the ``Process`` object begins the child fork, a
469-
single call to :meth:`_engine.Engine.dispose` will ensure any remaining
470-
connections are flushed. **This is the recommended approach**::
464+
2. Call :meth:`_engine.Engine.dispose` on any given :class:`_engine.Engine`
465+
**directly before** the new process is started, so that the new process
466+
will create new connections, as well as not attempt to close connections that
467+
were shared from the parent which can impact the parent's subsequent
468+
use of those connections. **This is the recommended approach**::
471469

472470
engine = create_engine("mysql+mysqldb://user:pass@host/dbname")
473471

474472
def run_in_process():
475-
# process starts. ensure engine.dispose() is called just once
476-
# at the beginning
473+
with engine.connect() as conn:
474+
conn.execute(text("..."))
475+
476+
# before process starts, ensure engine.dispose() is called
477+
engine.dispose()
478+
p = Process(target=run_in_process)
479+
p.start()
480+
481+
3. Alternatively, if the :class:`_engine.Engine` is only to be used in
482+
child processes, and will not be used from the parent process subsequent
483+
to the creation of child forks, the dispose may be within the child process
484+
right as it begins::
485+
486+
engine = create_engine("mysql+mysqldb://user:pass@host/dbname")
487+
488+
def run_in_process():
489+
# process starts. ensure engine.dispose() is called just once
490+
# at the beginning. note this cause parent process connections
491+
# to be closed for most drivers
477492
engine.dispose()
478493

479494
with engine.connect() as conn:
@@ -482,10 +497,14 @@ are three general approaches to this:
482497
p = Process(target=run_in_process)
483498
p.start()
484499

485-
3. An event handler can be applied to the connection pool that tests for
500+
# after child process starts, "engine" above should not be used within
501+
# the parent process for connectivity, without calling
502+
# engine.dispose() first
503+
504+
4. An event handler can be applied to the connection pool that tests for
486505
connections being shared across process boundaries, and invalidates them.
487506
This approach, **when combined with an explicit call to dispose() as
488-
mentioned above**, should cover all cases::
507+
mentioned above in options 2 or 3**, should cover all cases::
489508

490509
from sqlalchemy import event
491510
from sqlalchemy import exc

0 commit comments

Comments
 (0)