feat(sdk): add Amazon Redshift support to Workflow Insight#722
Merged
Conversation
…tension Adds Amazon Redshift as a queryable destination in the Workflow Insight VS Code extension, alongside the existing DynamoDB / Aurora / S3-Athena / CloudWatch Logs destinations. Redshift is modeled on the Aurora path (SQL over the same record_json + scalar-columns table) with the Redshift-specific differences handled: - New src/redshift.ts: runRedshiftQuery + fetchRedshiftRecord over the Redshift Data API. Unlike the RDS Data API, the Redshift Data API is asynchronous, so it executes then polls DescribeStatement to completion and pages through GetStatementResult, normalizing to the shared result shape. - schema.ts: new "redshift" DestinationType and a dedicated NL->SQL prompt (record schema, dialect, few-shots) covering SUPER/PartiQL navigation. JSON keys are camelCase, so the prompt requires double-quoted SUPER paths (record_json."functionName", o."durationMs") and array UNNEST via a joined table alias. - config.ts / package.json settings: redshift workgroup/cluster, database, schema, table, dbUser, and secretArn (Serverless or provisioned). Wired through extension.ts (query + record-fetch routing) and destinationTest.ts (SELECT 1 connectivity probe). Settings webview updated (SettingsModal, types, starter query). Adds @aws-sdk/client-redshift-data. Example CDK stack + exporter fixes (validated end-to-end against a live Redshift Serverless workgroup): - RedshiftExporter: rewrite the upsert MERGE to project the row in the USING subquery and join on src.execution_arn. The previous form joined on a bound parameter, which Redshift plans as a NestedLoop join and rejects with "NestedLoop join is not supported in MERGE" — so every export silently failed. - redshift-create-table handler: run multiple statements (splits on ';') so the provisioning SQL can CREATE TABLE and GRANT in one custom resource. - stack.ts: GRANT ALL on the table TO PUBLIC (the table is owned by the create-table role's DB user; the exporter and query identities are different users and were otherwise denied), set enable_case_sensitive_identifier=true on the workgroup so camelCase SUPER keys are reachable, and pass the Redshift env vars to the example function so it emits records via RedshiftExporter. Tests: unit tests for runRedshiftQuery/fetchRedshiftRecord (polling, pagination, failure, SUPER parse) and extended config/destinationTest coverage.
yaythomas
approved these changes
Jul 15, 2026
ParidelPooya
marked this pull request as ready for review
July 15, 2026 20:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Amazon Redshift as a queryable destination in the Workflow Insight VS Code extension, alongside the existing DynamoDB / Aurora / S3-Athena / CloudWatch Logs destinations. Redshift is modeled on the Aurora path (SQL over the same record_json + scalar-columns table) with the Redshift-specific differences handled:
Example CDK stack + exporter fixes (validated end-to-end against a live Redshift Serverless workgroup):
Tests: unit tests for runRedshiftQuery/fetchRedshiftRecord (polling, pagination, failure, SUPER parse) and extended config/destinationTest coverage.