fix: Bring back plugin validation #1108
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erezrokah
commented
Jul 21, 2023
| } | ||
|
|
||
| func (tt Tables) ValidateDuplicateTables() error { | ||
| tableNames := tt.TableNames() |
Member
Author
There was a problem hiding this comment.
Previous code only validated top level tables. Fixed it so it uses TableNames() which gives us relations too, and added a test
erezrokah
commented
Jul 21, 2023
| return fmt.Errorf("table name %q is not valid: table names must contain only lower-case letters, numbers and underscores, and must start with a lower-case letter or underscore", t.Name) | ||
| } | ||
| return nil | ||
| return ValidateTable(t) |
Member
Author
There was a problem hiding this comment.
There's some refactoring to be done around the validation code as we have a separate schema/validators.go file which also has some code to validate table names. It currently only validates the length, so I added the call here
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #1108 +/- ##
==========================================
+ Coverage 48.66% 48.87% +0.20%
==========================================
Files 86 87 +1
Lines 8012 8037 +25
==========================================
+ Hits 3899 3928 +29
+ Misses 3760 3749 -11
- Partials 353 360 +7
☔ View full report in Codecov by Sentry. |
disq
approved these changes
Jul 21, 2023
erezrokah
force-pushed
the
fix/validate_plugin
branch
from
71451d3 to
4d07755
Compare
yevgenypats
approved these changes
Aug 18, 2023
kodiakhq bot
pushed a commit
that referenced
this pull request
Aug 18, 2023
🤖 I have created a release *beep* *boop* --- ## [4.5.1](v4.5.0...v4.5.1) (2023-08-18) ### Bug Fixes * Bring back plugin validation ([#1108](#1108)) ([61765a7](61765a7)) * **deps:** Update module github.com/cloudquery/plugin-pb-go to v1.9.3 ([#1149](#1149)) ([e1ea578](e1ea578)) * **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.5.0 ([#1145](#1145)) ([70d12e4](70d12e4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
5 tasks
kodiakhq bot
pushed a commit
that referenced
this pull request
Sep 5, 2023
Follow up to #1108. This PR relaxes the validation to check only for duplicates. Some databases allow uppercase table tables and the validation in the SDK seemed to be PostgreSQL oriented. At some point we might want to have destination specific validation/normalization. Related to https://discord.com/channels/872925471417962546/873606591335759872/1148643989348679770 ---
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1106
Plugin validation was removed in https://github.com/cloudquery/plugin-sdk/pull/984/files#diff-95f04007d983a3bfdb080b338b9de5a8bbd73a3d65910fbe664f97788a9021b3.
As a result we don't validate tables and columns anymore, see resulted bug in cloudquery/cloudquery#12428.
This PR brings it back and per a better option puts the validation in the
Initmethod. Other options:.validateon the plugin side, probably after transformation here. Downside that each plugin needs to implement the callschema.TransformTableshere. Downside it will happen before any custom transformation that are executed on tables liketitleTransformerUse the following steps to ensure your PR is ready to be reviewed
go fmtto format your code 🖊golangci-lint run🚨 (install golangci-lint here)