(Postgres) Using the Psycopg 3 connection pool in SQLAlchemy #12522
Replies: 5 comments 14 replies
-
is it running a background thread? there's no other way to achieve that result.
oh, this is awkward. Can't psycopg's implementation simply do a "putconn" implicitly when we say you should use from sqlalchemy.dialects import registry
from sqlalchemy.dialects.postgresql.psycopg import PGDialect_psycopg
from sqlalchemy import NullPool
from sqlalchemy import create_engine
class MyDialect(PGDialect_psycopg):
@classmethod
def get_pool_class(cls, url):
return NullPool
def connect(self, *args, **params):
# this is your pool
return pool.getconn()
def do_close(self, dbapi_connection):
# this is your pool
pool.putconn(dbapi_connection)
# second argument is the module path, e.g. if we do `__import__(modulepath)`
registry.register("postgresql.mydialect", "__main__", "MyDialect")
e = create_engine("postgresql+mydialect://")
# this will be MyDialect
print(e.dialect)
# this will be NullPool
print(e.pool)if psycopg3 can make |
Beta Was this translation helpful? Give feedback.
-
|
@dvarrazzo - this works great for me too, thank you! Are these code comments correct? class MyConnection(psycopg.Connection):
def close(self):
if pool := getattr(self, "_pool", None):
# Connection was in use; return it to the pool
pool.putconn(self)
else:
# Connection was in pool; close it for real
super().close()@CaselIT - I would love to contribute back to the project! Would the example go in the Connection Pooling topic or somewhere else? |
Beta Was this translation helpful? Give feedback.
-
|
@CaselIT, @zzzeek - I opened #12540. Your feedback is welcomed. @dvarrazzo - here is the paragraph I used to introduce Psycopg 3's connection pooling - are there any additional features or differences that you'd like to be pointed out, or any revisions you'd like to make?
|
Beta Was this translation helpful? Give feedback.
-
|
Here are complete runnable examples for using a Psycopg 3 connection pool in SQLAlchemy (sans database URL).
|
Beta Was this translation helpful? Give feedback.
-
I had no idea that this was the "protocol" used by SQLAlchemy. Now that, for unrelated reasons, I'm starting to put together features for psycopg 3.3, I wouldn't mind at all to improve the integration. We could add a parameter like Would there be other changes in behaviour to make Psycopg better integrated with SQLAlchemy? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Psycopg 3 has a nice
psycopg_pool.ConnectionPoolimplementation (shout out to @dvarrazzo!) that can automatically disconnect unused pool connections from the server after a specifiedmax_idletime:psycopg_pool – Connection pool implementations > The ConnectionPool class
I would like to use
psycopg_pool.ConnectionPoolin my SQLAlchemy client application. There is some unfinished exploration of this in discussion #7478. Here, I would like to start a new discussion specifically focused on finding (and sharing) the solution.Here is what I have so far:
This is similar in structure to the Using cx_Oracle SessionPool example, except with an added
checkinevent listener to return the connection to the pool.do_stuff() #1succeeds:Note that in the last logging message,
NullPoolcloses the connection. I confirmed this by inspection of the_close_connection()method of thesqlalchemy.pool.Poolabstract class.do_stuff() #2obtains the same connection but the transaction fails because the connection is now closed:Is there a way to prevent
NullPoolfrom closing the connection? Why doesn't the cx_Oracle example have the same issue?Hopefully someone will see what I'm missing. Thank you in advance!
Beta Was this translation helpful? Give feedback.
All reactions