Skip to content

Index kg_edges by each node so connector deletes stop scanning it - #1639

Merged
helpcodeai merged 2 commits into
mainfrom
fix/kg-edges-node-indexes
Nov 11, 2026
Merged

helpcodeai merged 2 commits into
mainfrom
fix/kg-edges-node-indexes

Conversation

@helpcodeai

Copy link
Copy Markdown
Contributor

Summary

Deleting a connector took 3 to 7.6 s on Cloud. The connector's knowledge-graph nodes go with it, and for each node Postgres looks up kg_edges by source_node_id and by target_node_id. Neither column had an index of its own, so each node cost two full scans of kg_edges (about 261k rows). The Instagram delete path runs inside one transaction and hit Prisma's 5 s transaction timeout (Sentry ANYTHINGMCP-CLOUD-BACKEND-4, a connector with 30 nodes): the customer got a 500 and the connector stayed.

Fixes #1638. Fixes ANYTHINGMCP-CLOUD-BACKEND-4.

Changes

  • KgEdge: @@index([sourceNodeId]) and @@index([targetNodeId]).
  • Two migrations, one statement each: CREATE INDEX CONCURRENTLY IF NOT EXISTS. CONCURRENTLY does not lock the table for writes during the blue/green overlap (as deploy/cloud/release.sh asks for big tables), and it cannot run in a transaction block, hence one statement per migration. The index names are Prisma's defaults, so there is no drift.

Cloud and self-hosted: same migration, applied by prisma migrate deploy at backend start. The previous release runs fine against the new schema (indexes only).

Type

  • Bug fix
  • Performance improvement

Testing

  • Tested manually

In packages/backend, against a new empty database on the local Postgres (port 5433):

  • npx prisma migrate deploy: all migrations applied, including the two new ones (CONCURRENTLY accepted).
  • pg_indexes for kg_edges lists kg_edges_source_node_id_idx and kg_edges_target_node_id_idx.
  • npx prisma migrate diff --from-config-datasource --to-schema prisma/schema.prisma --exit-code: "No difference detected".

On Cloud (read-only): EXPLAIN ANALYZE of the lookup the cascade makes shows a parallel seq scan of kg_edges, 40-100 ms per node end.

No unit test: schema and SQL only, no code change.

Checklist

  • My code follows the existing code style
  • I have tested my changes locally
  • This PR has a descriptive title

Related Issues

Fixes #1638

@helpcodeai
helpcodeai merged commit 7d6395f into main Nov 11, 2026
15 of 16 checks passed
@helpcodeai
helpcodeai deleted the fix/kg-edges-node-indexes branch November 11, 2026 16:54
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Connector deletes take 3-7 s: kg_edges has no index on its node columns

1 participant