Skip to content

sql: don't drop table constraints after a nullable UNIQUE - #39598

Merged
antiguru merged 3 commits into
mainfrom
claude/gracious-keller-yg92qg
Oct 7, 2026
Merged

antiguru merged 3 commits into
mainfrom
claude/gracious-keller-yg92qg

Conversation

@antiguru

@antiguru antiguru commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

plan_create_table and plan_source_export_desc exited the whole constraints loop (break 'c) on a non-primary UNIQUE over a nullable column without NULLS NOT DISTINCT. Every later constraint was silently ignored. For example, CREATE TABLE t (a int, b int, UNIQUE (a), PRIMARY KEY (b)) lost its primary key and left b nullable. The fix skips only that constraint. The UNIQUE/PRIMARY KEY handling the two functions duplicated now lives in a shared helper.

Source purification: Postgres and MySQL always put the primary key first, so existing source exports never lost their PK or its implied NOT NULL. They could lose NOT NULL secondary unique keys that sort after a nullable UNIQUE (Postgres sorts by constraint OID, MySQL by index name). Those keys come back when the exports are replanned. Keys are not part of the persist schema. SQL Server marks every UNIQUE as NULLS NOT DISTINCT and was unaffected.

Regression test in test/sqllogictest/keys.slt, added in its own commit ahead of the fix.

Closes: SQL-759

🤖 Generated with Claude Code

https://claude.ai/code/session_01TqfshTuYavDY1L2GUsW5YW

claude added 2 commits October 6, 2026 19:08
Regression test for a non-primary UNIQUE constraint over a nullable column
causing all later constraints of a table to be ignored.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TqfshTuYavDY1L2GUsW5YW
A non-primary UNIQUE constraint over a nullable column without NULLS NOT
DISTINCT is not a key. Planning exited the whole constraints loop in that
case, dropping every later constraint, including a primary key and the
NOT NULL it implies. Skip just that constraint instead.

Factor the UNIQUE/PRIMARY KEY handling shared by plan_create_table and
plan_source_export_desc into plan_unique_constraints.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TqfshTuYavDY1L2GUsW5YW
@linear-code

linear-code Bot commented Oct 6, 2026

Copy link
Copy Markdown

SQL-759

keys.slt runs in standard mode, where each expected output line is one
value, not one row.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01TqfshTuYavDY1L2GUsW5YW
@antiguru
antiguru marked this pull request as ready for review October 7, 2026 06:43
@antiguru
antiguru requested a review from a team as a code owner October 7, 2026 06:43
@antiguru
antiguru requested a review from aljoscha October 7, 2026 06:43

@aljoscha aljoscha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks correct, but I don't know in detail what our constraints (ha!) here are

@antiguru
antiguru merged commit bc5f087 into main Oct 7, 2026
80 checks passed
@antiguru
antiguru deleted the claude/gracious-keller-yg92qg branch October 7, 2026 09:01
@def-

def- commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

QA LLM Review (Post Merge)

@antiguru — an automated review of commit bc5f0871f4 found the following potential MEDIUM+ issue(s) after this PR was merged.

1. HIGH -- Re-planning existing tables and source exports changes their persisted RelationDesc, crash-looping environmentd or the source replica on upgrade

src/sql/src/plan/statement/ddl.rs:1564

Existing items are re-planned from their stored create_sql at boot. Any table or source export whose constraints the old break 'c cut off now gets a different RelationDesc, with extra keys and possibly a PK column flipped to NOT NULL. That desc no longer matches the schema registered on its persist shard, so after upgrading, tables panic environmentd at boot and source exports crash-loop the replica that ingests them.

Details

Contrary to the commit message, keys are part of the persist schema. SourceData::encode_schema stores ProtoRelationDesc, which includes ProtoRelationType.keys. Persist looks up a writer's schema by full RelationDesc equality (src/persist-client/src/internal/apply.rs:262, src/persist-client/src/internal/state.rs:1622). On a mismatch it returns None once a schema already exists. Only MVs get a bootstrap schema evolution (evolve_nullability_for_bootstrap). Tables and source exports have none.

I reproduced both paths by stopping the parent commit's image (134006ec86) and starting this commit's image on the same /mzdata:

  • Table: CREATE TABLE t (a int, b int, UNIQUE (a), PRIMARY KEY (b)) is accepted without unsafe mode before this change, because keys stays empty and the unsafe_enable_table_keys gate is skipped. INSERT INTO t VALUES (1, NULL) also succeeds. The new image re-plans t as b NOT NULL, keys [[1]], because unsafe_enable_table_keys is enable_for_item_parsing. environmentd then aborts in register_table_collections at src/txn-wal/src/txns.rs:229 ("schema should be registered"), exit 134, on every restart. Even without the panic, the shard holds NULLs in a column now typed NOT NULL.
  • Postgres source: upstream s (a int, b int NOT NULL, UNIQUE (a), UNIQUE (b)) with CREATE TABLE s FROM SOURCE. Only the keys change ([] to [[1]]). clusterd panics in persist_sink::append_batches at src/persist-client/src/write.rs:745 ("fetching a schema id should only fail when the shard is tombstoned") about every 6s. That takes down the whole replica. The same applies to a PK plus a nullable UNIQUE plus a NOT NULL UNIQUE, a common upstream shape, and to MySQL.

From reading the code, 0dt does not catch this either. In read-only mode only migrated builtin tables are registered (src/storage-controller/src/lib.rs:1516) and sources do not append, so the failure appears only after promotion.

Suggested fix: keep the desc of existing items stable. Add a catalog migration that removes the UNIQUE/PRIMARY KEY table constraints the old planner never applied from stored create_sql, meaning every Unique constraint from the first nullable, non-NULLS NOT DISTINCT one onward. A per-export details flag, like cast_oid_full_range, would also work. Only newly created objects should get the fixed semantics. An upgrade check (platform-checks) with this table shape and this Postgres shape would cover it.

@antiguru

antiguru commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

Thanks for catching this. Reverting in #39660.

Correction to the description: keys are part of the persist schema. SourceData::encode_schema stores the full ProtoRelationDesc, keys included, and find_schema matches writer schemas by equality, so a change in keys alone is enough to miss the registered schema. Re-landing will keep existing items' descs stable and add platform-checks for the table and source shapes. Tracked in SQL-759.

🤖 Posted by Claude Code

antiguru added a commit that referenced this pull request Oct 7, 2026
…39660)

Reverts #39598

Restores old behavior, new behavior can change the schema, which would
result in a crash.
@bosconi bosconi mentioned this pull request Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants