feat(postgresql)!: Update to SDK V4 #11719
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erezrokah
commented
Jun 23, 2023
| pgType pgType | ||
| Tables schema.Tables | ||
| tables schema.Tables | ||
| cdcId string |
Member
Author
There was a problem hiding this comment.
Since plugins don't have access to the top level spec we can't use the source name as the replication name like before. I added a plugin level configuration for now, but we should figure this out before merging this PR as this is a breaking change
erezrokah
commented
Jun 23, 2023
| Unit: arrow.Millisecond, | ||
| } | ||
| err = timeMillisecond.Set("04:05:06.789") | ||
| err = inet.Set("192.168.0.0/24") |
Member
Author
There was a problem hiding this comment.
We can change this back to 192.168.0.1/24 after cloudquery/plugin-sdk#982 is released
erezrokah
commented
Jun 23, 2023
Member
Author
|
Marked this as blocked due to the breaking changes in the spec (adding a |
5 tasks
erezrokah
commented
Jun 23, 2023
erezrokah
force-pushed
the
fix/update_postgres_source_sdk_v4
branch
3 times, most recently
from
b4b53b3 to
b38df93
Compare
erezrokah
changed the title
Member
Author
Related to #11766. We'll need to chose the right naming for this plugin level configuration and document it |
hermanschaaf
changed the title
1 task
erezrokah
force-pushed
the
fix/update_postgres_source_sdk_v4
branch
2 times, most recently
from
2c92cf5 to
5068de7
Compare
erezrokah
force-pushed
the
fix/update_postgres_source_sdk_v4
branch
from
5068de7 to
af0287a
Compare
disq
reviewed
Jul 14, 2023
erezrokah
commented
Jul 14, 2023
|
|
||
| func (*Spec) Validate() error { | ||
| return nil | ||
| CDCId string `json:"cdc_id,omitempty"` |
Member
Author
There was a problem hiding this comment.
Updated the docs to reflect this
disq
approved these changes
Jul 14, 2023
erezrokah
added
automerge
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #11758
Summary
Similar to #11696.
I had to comment change some of the tests due to cloudquery/plugin-sdk#982 and cloudquery/plugin-sdk#915 (comment), so not ready to mergeBEGIN_COMMIT_OVERRIDE
feat!: Upgrades the postgresql source plugin to use plugin-sdk v4. This version does not contain any user-facing breaking changes, but because it is now using CloudQuery gRPC protocol v3, it does require use of a destination plugin that also supports protocol v3. All recent destination plugin versions support this.
feat!: To enable CDC in this version you'll need to use the
cdc_idconfiguration string property, instead of thecdcboolean one. Please refer to the docs for more informationEND_COMMIT_OVERRIDE