Skip to content

Move to sqlparser 0.64 and drop the sqlparser fork #264

Closed

Description

@zvonimir-dd

What

Move from our sqlparser fork to the sqlparser 0.64.0 release, and delete the [patch.crates-io] entry that pulls the fork from git.

Why

  • crates.io drops [patch] from a published manifest. So as long as the fork is patched in, a published sqleq would build against stock sqlparser, which lacks the fork's fields, and would not compile. This blocks the release.
  • 0.64.0 ships both of the fork's fixes, Postgres: support OVERRIDING { SYSTEM | USER } VALUE in INSERT apache/datafusion-sqlparser-rs#2626 (OVERRIDING { SYSTEM | USER } VALUE) and #2627 (the ON CONFLICT target's WHERE index_predicate). The fork has no other reason to exist.

What 0.64 changes for us

A trial bump gives 33 compile errors:

Change Where we use it
Expr::InList and Expr::InSubquery become one Expr::In { expr, target: InTarget, negated, global }. InTarget also has Unnest and a new Table (SQLite's IN tablename); global is ClickHouse's GLOBAL IN. the frontend (lower.rs, normalize.rs, infer.rs, params.rs, dml.rs); the fuzz axis (typing.rs, literals.rs)
The ON CONFLICT predicate is ConflictTarget::Columns { columns, predicate }, not the fork's OnConflict.conflict_target_predicate the Lean axis (idents.rs, translate.rs, recognize.rs)
TableFactor::Table: index_hints is renamed index_directives, and a new inheritance: Option<TableInheritance> holds Postgres's ONLY t and t * lower.rs, dml.rs
New variants: JoinConstraint::UsingAlias (JOIN ... USING (cols) AS alias), ArrayElemTypeDef::ParenthesisNotNull, AlterColumnOperation::GenericOptions the fuzz axis (limits.rs), normalize.rs, catalog.rs, the Lean axis
ObjectName lost a field one call site reads collation.rs

Insert.overriding is the fork's field, unchanged, so the two OVERRIDING refusals stand as they are.

Approach

Port with no behaviour change:

  1. Bump to sqlparser = { version = "0.64", features = ["visitor"] } and delete [patch.crates-io].

  2. Map the old shapes onto the new ones: InList becomes In { target: List }, InSubquery becomes In { target: Subquery }, Columns(cols) becomes Columns { columns, .. }, and the fork's predicate check becomes Columns { predicate: Some(_), .. }.

  3. Refuse what 0.64 newly parses unless it means what we already lower:

    • IN <table> and GLOBAL IN;
    • FROM ONLY t and t *;
    • JOIN ... USING (...) AS alias;
    • the new array element form;
    • ALTER COLUMN ... OPTIONS.

    0.63 could not parse any of these, so each was already refused. Supporting one is a separate change, with its own pins.

  4. Audit every catch-all arm (_ =>) over a type that gained a variant or field between 0.63 and 0.64. Each new variant must reach a refusal, never a silent fall-through.

Soundness

A parser upgrade changes what the frontend lowers only through the tree it is given. Two things keep the tree's meaning unchanged:

  • every new shape is either mapped onto the old one it replaces (the same SQL, re-spelled), or refused;
  • no catch-all arm accepts a new variant.

Statements that newly parse were refused before and are refused now, so their refusal kind does not change: Postgres's own grammar decides parse against unsupported, not sqlparser.

Acceptance

  • sqleq-check --expect pinned is byte-identical before and after on frontend, fuzz, sqleq-solver, qed, lean and cvc5. That includes the three pairs that needed the fork: dml/overriding_system_vs_user, insert_unnest/overriding_value and insert_unnest/conflict_target_predicate.
  • New pins: one for each newly parsed construct listed above, refused by the frontend.
  • The frontend's lowered IR is identical per case over a large differential set. Any case that moves is attributed to a named 0.64 change.
  • Cargo.toml has no git dependency and no [patch]; cargo deny check and the licence CSV check pass.
  • CI green, including MSRV.

Dependencies

None. It unblocks the packaging check in the release tracking issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions