Orchestrate all the pieces for online resharding with a few admin commands, including moving application traffic from old to new sharded cluster. Includes support for rollbacks with reverse logical replication.

Individual changes

• Added PostCutover step (internal) to schema sync to make the old shards suitable for reverse logical replication (e.g., remove WITH GENERATED ALWAYS identity constraint)
• Added a bunch of admin database commands:
  • SCHEMA_SYNC [pre|post] <source> <destination> <publication>
  • COPY_DATA <source> <destination> <publication> [<replication slot>]
  • REPLICATE <source> <destination> <publication> [<replication slot>]: only setup replication, nothing else. Data isn't synchronized, this is mostly for testing but can be used to create logical replicas that don't need synchronization or which don't currently accept writes
  • SHOW REPLICATION_SLOTS: show replication slots managed by us
  • SHOW SCHEMA_SYNC: track long running CREATE INDEX-style queries
  • SHOW TABLE_COPIES: track long-running initial table data sync, with progress details, e.g., rows, bytes written, etc.
  • SHOW TASKS: admin mostly, shows which one of any of the above are running in the background
  • STOP TASK <id>: cancel any of the background tasks
  • CUTOVER: perform online traffic cutover between old and new shards
  • RESHARD: do the resharding in one swoop. Equivalent of running SCHEMA_SYNC PRE, COPY_DATA, SCHEMA_SYNC POST, and CUTOVER manually.
• Added reverse logical replication for the cutover step. This can be used for rollbacks. A rollback can be done by running CUTOVER again. There is no data loss, since reverse replication is setup while traffic is paused.