Skip to content

Commit 989cb7c

Browse files
SQUASH – review comment: cite the tracker issue beside every documented limitation
docs: document DELETE and UPDATE for SQL users and table provider authors PR apache#19142 added `TableProvider::delete_from()` and `TableProvider::update()`, and implemented both for `MemTable`, but added no documentation. Add a `DELETE` section and an `UPDATE` section to the SQL user guide, with the syntax, the result shape, which table kinds support the statements, and the current limitations. Add a "Row-Level DML" section to the custom table provider guide, covering what the planner passes to each hook, the `count` result contract, the semantic rules a provider must follow, and a compiling example. Two behaviours found while verifying the documentation are recorded as warnings, since users meet them today: - An `IN` or an `EXISTS` subquery in the `WHERE` clause makes the statement apply to all rows, because the optimizer rewrites the subquery into a join and the predicate never reaches the provider. - `EXPLAIN DELETE` and `EXPLAIN UPDATE` execute the statement on an in-memory table, because `MemTable` changes the rows inside the hook and the hook runs during physical planning. Every documented limitation ends with the issue that tracks it, in one phrasing a contributor can grep for, so a merged fix makes the obsolete paragraph easy to find: apache#24654 for the subquery cases, apache#24656 for `EXPLAIN`, apache#24998 for the ignored `LIMIT` on a `DELETE`, and apache#19950 for `UPDATE ... FROM`. Which tables support the statements is a capability rather than a defect, so those two lines cite nothing. Assisted-by: Claude Code:claude-opus-5
1 parent ca7e59b commit 989cb7c

2 files changed

Lines changed: 9 additions & 9 deletions

File tree

‎docs/source/library-user-guide/custom-table-providers.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -910,14 +910,14 @@ impl TableProvider for MyMutableTable {
910910

911911
The planner drops some clauses before it calls a hook, so do not expect them:
912912

913-
- A `LIMIT` clause on a `DELETE` has no effect. Your hook sees only the filters.
914-
- A scalar subquery in a `WHERE` clause or in a `SET` clause fails during physical planning, before the hook runs.
915-
- An `IN` or an `EXISTS` subquery reaches your hook with an empty `filters` vector, because the optimizer rewrites the subquery into a join. Your hook then changes every row, which is the wrong answer. DataFusion does not yet protect a provider against this case.
916-
- `UPDATE ... FROM`, which reads new values from a second table, returns a "not implemented" error.
913+
- A `LIMIT` clause on a `DELETE` has no effect. Your hook sees only the filters. This limitation is tracked at [issue #24998](https://github.com/apache/datafusion/issues/24998).
914+
- A scalar subquery in a `WHERE` clause or in a `SET` clause fails during physical planning, before the hook runs. This limitation is tracked at [issue #24654](https://github.com/apache/datafusion/issues/24654).
915+
- An `IN` or an `EXISTS` subquery reaches your hook with an empty `filters` vector, because the optimizer rewrites the subquery into a join. Your hook then changes every row, which is the wrong answer. DataFusion does not yet protect a provider against this case. This limitation is tracked at [issue #24654](https://github.com/apache/datafusion/issues/24654).
916+
- `UPDATE ... FROM`, which reads new values from a second table, returns a "not implemented" error. This limitation is tracked at [issue #19950](https://github.com/apache/datafusion/issues/19950).
917917

918918
### When the Work Happens
919919

920-
The hooks run during physical planning, like `scan()`. A hook that changes rows before it returns its plan therefore changes them during planning, and `EXPLAIN DELETE` or `EXPLAIN UPDATE` also changes them. [MemTable] works this way.
920+
The hooks run during physical planning, like `scan()`. A hook that changes rows before it returns its plan therefore changes them during planning, and `EXPLAIN DELETE` or `EXPLAIN UPDATE` also changes them. [MemTable] works this way. This limitation is tracked at [issue #24656](https://github.com/apache/datafusion/issues/24656).
921921

922922
For a provider that writes to durable storage, do the work in the `execute()` method of the plan that you return instead. The hook then stays lightweight, and `EXPLAIN` shows the plan without a side effect.
923923

‎docs/source/user-guide/sql/dml.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -237,13 +237,13 @@ This feature is not implemented: DELETE not supported for Base table
237237
### Limitations
238238

239239
:::{warning}
240-
Do not use a subquery in the condition of a `DELETE` or an `UPDATE`. A scalar subquery, such as `WHERE id = (SELECT max(id) FROM other)`, returns an error. An `IN` or an `EXISTS` subquery is worse: the statement applies to **all** rows of the table. The optimizer rewrites the subquery into a join, and the condition then no longer reaches the table provider.
240+
Do not use a subquery in the condition of a `DELETE` or an `UPDATE`. A scalar subquery, such as `WHERE id = (SELECT max(id) FROM other)`, returns an error. An `IN` or an `EXISTS` subquery is worse: the statement applies to **all** rows of the table. The optimizer rewrites the subquery into a join, and the condition then no longer reaches the table provider. This limitation is tracked at [issue #24654](https://github.com/apache/datafusion/issues/24654).
241241
:::
242242

243243
:::{warning}
244-
`EXPLAIN` executes a `DELETE` or an `UPDATE` on an in-memory table. The provider changes the rows while DataFusion plans the statement. Use a copy of the table if you want to read the plan only.
244+
`EXPLAIN` executes a `DELETE` or an `UPDATE` on an in-memory table. The provider changes the rows while DataFusion plans the statement. Use a copy of the table if you want to read the plan only. This limitation is tracked at [issue #24656](https://github.com/apache/datafusion/issues/24656).
245245
:::
246246

247-
DataFusion ignores a `LIMIT` clause in a `DELETE` statement. The statement removes all rows that match the condition.
247+
DataFusion ignores a `LIMIT` clause in a `DELETE` statement. The statement removes all rows that match the condition. This limitation is tracked at [issue #24998](https://github.com/apache/datafusion/issues/24998).
248248

249-
`UPDATE ... FROM`, which reads the new values from a second table, returns a "not implemented" error. See [issue #19950](https://github.com/apache/datafusion/issues/19950).
249+
`UPDATE ... FROM`, which reads the new values from a second table, returns a "not implemented" error. This limitation is tracked at [issue #19950](https://github.com/apache/datafusion/issues/19950).

0 commit comments

Comments
 (0)