Skip to content

Implement remote libSQL DDL support and refactor DDL builders - #4

Merged
jonaspm merged 16 commits into
mainfrom
feat/alter
Aug 19, 2026
Merged

jonaspm merged 16 commits into
mainfrom
feat/alter

Conversation

@jonaspm

@jonaspm jonaspm commented Aug 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Implements remote DDL support (ALTER COLUMN, foreign key mutations) for Turso/sqld connections via the libSQL fork, rewrites the client to use the embedded libSQL fork of SQLite (replacing the rusqlite driver), and lands a round of contract fixes to match the host's RpcDriver expectations.

Changes

  • libSQL fork everywhere — local connections now use the embedded libsql crate (core feature, bundled) instead of rusqlite, so fork extensions (ALTER COLUMN, FK add/drop) work on local files and remote servers alike. No async runtime: bridged with futures::executor::block_on.
  • Remote DDL — get_alter_column_sql, get_create_foreign_key_sql, drop_foreign_key now work against remote backends, gated by a new is_remote check on Client (src/client.rs). The FK builder emits the libSQL ALTER COLUMN col TO col <type> REFERENCES ... form, introspecting the column type via PRAGMA table_info (host now passes connection params through for this method).
  • DDL builder refactor — SQL builders in src/handlers/ddl.rs rewritten as pure functions that don't require a connection; DDL responses consistently return arrays of SQL strings.
  • Metadata contract fixes — column keys renamed is_primary_key → is_pk, column_default → default_value; unused comment field removed. Test added to pin the contract keys.
  • connection_uri support — new ConnectionParams.connection_uri field; resolve_backend now treats the raw URI as authoritative over decomposed host/password fields, preserving auth tokens. .tabularium advertises connection_uri/connection_uri_schemes. Tests cover precedence, backend resolution, and token preservation.
  • get_triggers RPC method — new handler, enabled with the other metadata changes.
  • CRUD fix — insert returns affected rows count instead of null.
  • .tabularium capabilities — alter_column: true, create_foreign_keys: true kept in sync with actual behavior.
  • Docs — README updated for the remote/local behavior split and host contracts.

Testing

  • cargo test (unit tests incl. new FK builder, local unsupported error, contract keys, connection URI resolution)
  • cargo clippy --all-targets -- -D warnings
  • cargo fmt --all

jonaspm added 6 commits August 7, 2026 21:41
Enable `ALTER COLUMN` and foreign key mutations for remote Turso/sqld
connections using the libSQL fork. Local SQLite connections continue to
return a clear unsupported error.

- Add `is_remote` check to `Client` to gate libSQL-specific features
- Implement `get_alter_column_sql`, `get_create_foreign_key_sql`, and
  `drop_foreign_key` for remote backends
- Update `.tabularium` capabilities for `alter_column` and
  `create_foreign_keys`
- Update README and metadata handlers to align with host contracts
- Ensure DDL responses return arrays of SQL strings
Refactor `src/handlers/ddl.rs` to implement SQL builders as pure
functions
that do not require database connections.

Disable `create_foreign_keys` in `.tabularium` and return an unsupported
error in `get_create_foreign_key_sql`, as the host protocol does not
provide the column type necessary for libSQL's `ALTER COLUMN` rewrite.

Update README and internal documentation to clarify the distinction
between remote libSQL extensions and vanilla SQLite limitations.
Rename `is_primary_key` to `is_pk` and `column_default` to
`default_value`, and remove the unused `comment` field. Add a test
to verify the contract keys.
`create_foreign_keys` capability in `.tabularium`. Update the
`get_create_foreign_key_sql` handler to receive and use connection
params, and emit the libSQL `ALTER COLUMN` form. Add tests for the new
foreign key builder and local unsupported error.
`resolve_backend` logic to use the raw URI when available, allowing it
to be authoritative over decomposed host and password fields. Add tests
covering `connection_uri` precedence, URL vs. local backend resolution,
and token preservation. Update `.tabularium` capabilities to support
`connection_uri` and `connection_uri_schemes`.
@jonaspm
jonaspm marked this pull request as draft August 10, 2026 23:17
code to use the embedded libSQL fork instead of the rusqlite driver.

Changes:
- `Cargo.toml`: replace `rusqlite` with `libsql = { version = "0.9",
  default-features = false, features = ["core"] }`, add `futures`,
  update description to mention the fork, `core` feature, async/sync
  bridge
- `src/client.rs`: switch to `libsql::Connection` and `libsql::Value`,
  adapt all local connections, `open_local`, `local_query`,
  `json_to_libsql`, `libsql_value_to_json`, and rename `is_remote`
  comment
- `src/error.rs`: change `From<rusqlite::Error>` to
  `From<libsql::Error>`
- `src/handlers/ddl.rs`: remove `require_remote` (local works through
  fork), update docs for `get_create_foreign_key_sql`,
  `drop_foreign_key`, and `drop_index`, update
  `get_create_foreign_key_sql` test to use temp file
- `src/handlers/metadata.rs`: adjust formatting of a wrapped `connect`
  call
- `src/rpc.rs`: rename test to describe local fork working through
  `get_create_foreign_key_sql`
- `src/utils/values.rs`: update comment to say libsql (local) instead of
  rusqlite (local)
@jonaspm

jonaspm commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author
image @debba one thing I haven't included here is a fix to the justfile for the windows dev-install script but also for the other OS installation locations.

@jonaspm
jonaspm requested a review from debba August 11, 2026 19:06
@jonaspm
jonaspm marked this pull request as ready for review August 11, 2026 19:06
@debba

debba commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Review — PR #4

Overall the PR is solid on the host-contract side (payload shapes, metadata key renames, URI scheme passthrough were all checked against the tabularis host source and hold up). However, 5 real bugs were confirmed empirically against the PR's own embedded libSQL fork, plus a manifest issue and a few smaller items.

Confirmed bugs (should be fixed before merge)

1. file:/// URIs on Unix silently open the wrong database — src/client.rs:287
expand_path strips the entire file:/// prefix, so file:///data/app.db (the exact form shown in the manifest's own connection_string_example) becomes the relative path data/app.db. Builder::new_local then silently creates/opens an empty ./data/app.db instead of /data/app.db. The previous code stripped only file:, leaving ///data/app.db, which POSIX resolves absolutely. The only test covers the Windows file:///C:/... form, where dropping the third slash happens to be correct.

2. FK add/drop strips NOT NULL and DEFAULT from the rewritten column — src/handlers/ddl.rs:398
The column definition is rebuilt from PRAGMA table_info using only the type (cell 2), ignoring notnull (3) and dflt_value (4) from the same row. Verified: on user_id INT NOT NULL DEFAULT 7 REFERENCES users(id), after drop_foreign_key the column is just INT — both constraints are gone. Same applies to get_create_foreign_key_sql.

3. Dropping a composite FK reports success while doing nothing — src/handlers/ddl.rs:420
For FOREIGN KEY(a,b) REFERENCES p(x,y), rewriting a single column doesn't touch the table-level constraint: PRAGMA foreign_key_list still shows both rows afterwards, but the handler returns Ok, so the host UI shows the FK as dropped.

4. Changing a column's type/default/nullability destroys its existing FK — src/handlers/ddl.rs:221
Same root cause as #2: ALTER COLUMN x TO x <def> replaces the entire column definition, but build_alter_column_sql rebuilds it only from the dialog's fields. Retyping user_id INT REFERENCES users(id) to BIGINT silently drops the REFERENCES clause (and any column-level UNIQUE/CHECK).

5. Simultaneous rename + retype silently discards the retype — src/handlers/ddl.rs:213
When names differ, the builder early-returns with only RENAME COLUMN, dropping any type/default/NOT NULL changes with no error. Since the handler already returns Vec<String>, it could emit the rename followed by the ALTER COLUMN with no contract change.

Bugs 2, 4 and 5 share one root cause: the ALTER COLUMN builders never carry the column's full existing definition through the rewrite.

Manifest icon doesn't resolve anywhere

.tabularium sets "icon": "sqlite", but the Tabularium spec defines icon as a path or URL to a square logo, so the registry renders nothing. In the host app, EngineCard.tsx resolves symbolic icon names only for builtin drivers — third-party plugins fall back to the generic database glyph. Fix: ship a square logo in the repo (e.g. assets/icon.png, as the CSV plugin does) and point the manifest at it. (FYI: the BigQuery plugin has the same bug with "icon": "cloud".)

Other findings

  • Fragile trigger parser — src/handlers/metadata.rs:138: timing/event are classified via contains() over the whole trigger SQL, so body keywords override the header. The common audit trigger AFTER UPDATE ... BEGIN INSERT INTO log ... reports event=INSERT; the PR's own test passes only by luck of branch order. Parse the header before the ON keyword instead.
  • README contradicts the manifest and the code — README.md:79 (and lines 55–57): it claims create_foreign_keys is false and that local ALTER COLUMN / FK drop fail, while .tabularium sets true and the PR's own tests (local_alter_column_retypes_through_the_fork, drop_foreign_key_local_rewrites_the_schema) prove local support works.
  • Duplicated FK-name scheme — the fk_{table}_{from}_{id} format is hardcoded both in ddl.rs:430 (reverse-mapping) and metadata.rs:63 (generation). If either changes, drop_foreign_key silently stops matching every FK the plugin itself reports. Extract a shared fk_name(table, column, id) helper.
  • Full futures facade for a single block_on — Cargo.toml:26: use futures = { version = "0.3", default-features = false, features = ["executor"] } (or futures-executor directly) to cut the dependency tree and compile time.
  • block_on per row instead of per query — src/client.rs:102: the fetch loop enters a fresh executor for every row (10k rows = 10k+1 waker/parker setups on the hottest path). Wrap the whole loop in a single block_on(async { ... }).

@debba

debba commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

image @debba one thing I haven't included here is a fix to the justfile for the windows dev-install script but also for the other OS installation locations.

Thanks for this, it was a real issue, but I merged this TabularisDB/tabularis#258 in latest release, so probably won't work now :D

Receipt

- src/client.rs: 285-315 - platform-aware file: strip; Unix keeps /
  root, Windows keeps drive form.
- src/client.rs:415-473 - strip unit tests + platform-split file-URI
  backend tests.
- src/handlers/ddl.rs: 185-209 - get_alter_column_sql optional schema
  path when params present; pure builder
  fallback.
- src/handlers/ddl.rs: 211-389 - ColumnRewrite merge old+new defs;
  rename+retype emits both statements; schema
  path preserves REFERENCES/ON UPDATE/DELETE.
- src/handlers/ddl.rs: 393-500 - FK add/drop carry NOT NULL + DEFAULT
  via ColumnDef/column_def_sql.
- src/handlers/ddl.rs:506-573 - composite FK drop returns error (fork
  has no DROP CONSTRAINT, per libsql docs);
  no more silent no-op.
- src/handlers/ddl.rs tests - 12 new/updated (89 total pass).
Header scan stops at BEGIN; body keywords (e.g. audit trigger INSERTing into a log) no longer override the event.
Local files run the embedded libSQL fork, so retype, FK add/drop and constraint-preserving rewrites work everywhere; README previously claimed they were Turso-only.
@jonaspm

jonaspm commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

@debba Addressed all feedback items.

Full futures facade for a single block_on — Cargo.toml:26: use futures = { version = "0.3", default-features = false, features = ["executor"] } (or futures-executor directly) to cut the dependency tree and compile time.

Made the change but goal unmet. cargo tree -i futures shows libsql core feature hard-requires dep:futures (core = [..., "dep:futures", ...] in libsql 0.9.30). Facade still compiles; futures-executor was already in tree via libsql's facade. Swap = dependency accuracy only (we use one symbol, not the facade), no compile-time or tree savings.

@jonaspm

jonaspm commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

image @debba one thing I haven't included here is a fix to the justfile for the windows dev-install script but also for the other OS installation locations.

Thanks for this, it was a real issue, but I merged this TabularisDB/tabularis#258 in latest release, so probably won't work now :D

Fixed: Update installation paths

@debba

debba commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

@jonaspm You’re right, thanks for verifying it. Since libsql already requires the futures facade, my claim about reducing the dependency tree and compile time doesn’t hold here. Using futures-executor is still more precise for our direct usage, but brings no practical build savings. Consider this item resolved. If you agree, I think we’re ready to merge and release the plugin.

@jonaspm

jonaspm commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

Merging 🎉

@jonaspm
jonaspm merged commit ed8c4df into main Aug 19, 2026
@jonaspm
jonaspm deleted the feat/alter branch August 20, 2026 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants