Skip to content

Move Relational API to BindStatement - #697

Merged
evertlammerts merged 4 commits into
v2.0-cyanopterafrom
relational-sql
Oct 20, 2026
Merged

evertlammerts merged 4 commits into
v2.0-cyanopterafrom
relational-sql

Conversation

@evertlammerts

@evertlammerts evertlammerts commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Core is replacing its C++ Relation classes with one primitive, ClientContext::BindStatement. The Python relational API was built on those classes, bound eagerly through them, and could not express a lazy parameterised relation. This PR replaces that layer. It's a port that also changes behavior.

A relation is now a value in Python: a step record, its inputs, a schema bound once when the step is built, its parameters, and handles on the Python objects it reads. It runs nothing until a terminal. Every step renders into one SELECT while SQL clause order allows, otherwise it closes that SELECT as a common table expression; sql_query() is the text that runs. Names and types are whatever the engine's binder says. User SQL fragments are validated by the engine's parser and spliced as written.

What changes for users:

  • A bare string operand of an expression is a value, not a column (col("s") == "x" compares with the text). A warning for one release when it matches a column name.
  • A parameterised sql() is lazy and binds its values on every read.
  • After a join, column names are the engine's; an input alias stops resolving once a step started a new SELECT.
  • distinct().project() does what it says; a filter after a limit applies after the limit.
  • rel.query(name, sql) leaves no temporary view behind.
  • A SELECT the engine classifies as writing (nextval) runs at the call.
  • A persistent view over held rows (materialize(), values()) is refused; use rel.create() or con.register().
  • New verbs: with_columns, drop, rename, unnest, unpivot, pivot, sample, head, offset, union_by_name, group_by().agg(), join(suffix=).

Deleted: DuckDBPyRelation and the expression classes in C++, every include of the engine's relation headers.

Notes:

  • Construction costs about 0.3 ms per step, mostly the 2.0 parser; data paths are unchanged.
  • The behavior contract with every deliberate difference is BEHAVIOR.md on the branch.

Next PRs:

  • Refuse a DB-API call that re-enters a fetch on the same connection.
  • One value conversion path through the native converter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant