Repository navigation
Commit f199d53
authored
Keep import duckdb from opening a database and fix the default connection race (duckdb#680)
Fixes duckdb#612.
Importing duckdb opened the default in-memory database and started its
thread pools, because the DB API NUMBER type object parsed the strings
"BIGNUM" and "DECIMAL" at module scope, and parsing a type string
without a connection runs on the default connection. Logical types
belong to a connection's catalog, so the DB API categories no longer
hold type instances at all: STRING, NUMBER, DATETIME and BINARY are sets
of type ids, and `column_type == duckdb.NUMBER` tests the id of the type
a connection's result handed back. Import now starts no thread.
Consequences:
- Every DECIMAL is a NUMBER. Before, only DECIMAL(18,3) matched, because
"DECIMAL" parsed to that one width and scale.
- A JSON column is a STRING (its id is varchar), and an aggregate state
matches by the id of its value.
- `DBAPITypeObject` takes a collection of type ids; its `types` list is
gone.
Comparing a column type with a DB API type object, column type on the
left, printed a nanobind implicit conversion failure to stderr on every
call. A third equality overload handles that operand; every equality
result is unchanged.
Removing the import-time connection exposed a pre-existing deadlock: the
default connection holder kept its mutex while connecting, and
connecting releases the GIL, so two threads whose first `duckdb.sql()`
created the default connection hung the process (before, only
`duckdb.close()` followed by such a race reached it).
`set_default_connection` racing a reader deadlocked the same way, on
1.5.6 too. The holder now keeps its mutex only around pointer reads and
swaps; connecting and destroying happen outside it. A finalizer that
fetches the default connection no longer deadlocks when the default is
replaced, and once the module's cleanup has run while the interpreter
finalizes, the holder neither opens nor keeps a connection, so nothing
is left open at exit. A first-use race may briefly open one database per
racing thread.5 files changed
Lines changed: 643 additions & 57 deletions
File tree
- duckdb
- src
- include/duckdb_python/pyconnection
- typing
- tests/fast/api
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
29 | 31 | | |
30 | 32 | | |
31 | 33 | | |
| |||
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
38 | | - | |
39 | | - | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
40 | 43 | | |
41 | 44 | | |
42 | | - | |
| 45 | + | |
43 | 46 | | |
44 | 47 | | |
45 | | - | |
46 | | - | |
47 | | - | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
48 | 51 | | |
49 | 52 | | |
50 | 53 | | |
51 | 54 | | |
52 | 55 | | |
53 | 56 | | |
54 | | - | |
| 57 | + | |
55 | 58 | | |
56 | 59 | | |
57 | 60 | | |
58 | | - | |
| 61 | + | |
59 | 62 | | |
60 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
61 | 67 | | |
62 | 68 | | |
63 | 69 | | |
64 | 70 | | |
65 | 71 | | |
66 | | - | |
67 | | - | |
| 72 | + | |
| 73 | + | |
68 | 74 | | |
69 | 75 | | |
70 | 76 | | |
| |||
73 | 79 | | |
74 | 80 | | |
75 | 81 | | |
76 | | - | |
77 | | - | |
| 82 | + | |
| 83 | + | |
78 | 84 | | |
79 | | - | |
80 | | - | |
81 | | - | |
| 85 | + | |
82 | 86 | | |
83 | 87 | | |
84 | 88 | | |
85 | 89 | | |
86 | 90 | | |
87 | | - | |
| 91 | + | |
88 | 92 | | |
89 | 93 | | |
90 | 94 | | |
91 | 95 | | |
92 | 96 | | |
93 | | - | |
| 97 | + | |
94 | 98 | | |
95 | 99 | | |
96 | 100 | | |
97 | 101 | | |
98 | | - | |
| 102 | + | |
99 | 103 | | |
100 | 104 | | |
101 | 105 | | |
102 | 106 | | |
103 | | - | |
| 107 | + | |
104 | 108 | | |
105 | 109 | | |
106 | 110 | | |
| |||
113 | 117 | | |
114 | 118 | | |
115 | 119 | | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
132 | 136 | | |
133 | 137 | | |
134 | 138 | | |
| |||
144 | 148 | | |
145 | 149 | | |
146 | 150 | | |
147 | | - | |
| 151 | + | |
148 | 152 | | |
149 | 153 | | |
150 | 154 | | |
| |||
161 | 165 | | |
162 | 166 | | |
163 | 167 | | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
173 | | - | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
174 | 178 | | |
175 | 179 | | |
176 | 180 | | |
| |||
201 | 205 | | |
202 | 206 | | |
203 | 207 | | |
204 | | - | |
| 208 | + | |
205 | 209 | | |
206 | 210 | | |
207 | 211 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
52 | 55 | | |
53 | 56 | | |
54 | 57 | | |
| 58 | + | |
55 | 59 | | |
56 | 60 | | |
57 | 61 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2155 | 2155 | | |
2156 | 2156 | | |
2157 | 2157 | | |
| 2158 | + | |
| 2159 | + | |
| 2160 | + | |
| 2161 | + | |
| 2162 | + | |
| 2163 | + | |
| 2164 | + | |
| 2165 | + | |
| 2166 | + | |
| 2167 | + | |
| 2168 | + | |
| 2169 | + | |
| 2170 | + | |
| 2171 | + | |
| 2172 | + | |
| 2173 | + | |
| 2174 | + | |
| 2175 | + | |
| 2176 | + | |
| 2177 | + | |
| 2178 | + | |
| 2179 | + | |
| 2180 | + | |
2158 | 2181 | | |
2159 | | - | |
2160 | | - | |
2161 | | - | |
2162 | | - | |
| 2182 | + | |
| 2183 | + | |
| 2184 | + | |
| 2185 | + | |
| 2186 | + | |
| 2187 | + | |
| 2188 | + | |
| 2189 | + | |
| 2190 | + | |
| 2191 | + | |
| 2192 | + | |
| 2193 | + | |
| 2194 | + | |
| 2195 | + | |
| 2196 | + | |
| 2197 | + | |
| 2198 | + | |
| 2199 | + | |
| 2200 | + | |
| 2201 | + | |
| 2202 | + | |
2163 | 2203 | | |
2164 | | - | |
| 2204 | + | |
2165 | 2205 | | |
2166 | 2206 | | |
2167 | 2207 | | |
2168 | | - | |
2169 | | - | |
| 2208 | + | |
| 2209 | + | |
| 2210 | + | |
| 2211 | + | |
| 2212 | + | |
| 2213 | + | |
| 2214 | + | |
| 2215 | + | |
| 2216 | + | |
| 2217 | + | |
| 2218 | + | |
| 2219 | + | |
| 2220 | + | |
| 2221 | + | |
| 2222 | + | |
| 2223 | + | |
| 2224 | + | |
| 2225 | + | |
| 2226 | + | |
| 2227 | + | |
| 2228 | + | |
2170 | 2229 | | |
2171 | 2230 | | |
2172 | 2231 | | |
| |||
2519 | 2578 | | |
2520 | 2579 | | |
2521 | 2580 | | |
2522 | | - | |
| 2581 | + | |
2523 | 2582 | | |
2524 | 2583 | | |
2525 | 2584 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
364 | 364 | | |
365 | 365 | | |
366 | 366 | | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
367 | 380 | | |
368 | 381 | | |
369 | 382 | | |
| |||
0 commit comments