feat!: Add ability to override CSV File options (headers and delimiters) #6958
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bbernays
requested review from
a team and
hermanschaaf
and removed request for
a team
candiduslynx
suggested changes
Jan 18, 2023
Contributor
candiduslynx
left a comment
candiduslynx
left a comment
There was a problem hiding this comment.
I wonder if we could also add CLRF support to the csv plugin, as encoding/csv.Writer supports it, too. (Bound to another PR).
candiduslynx
approved these changes
Jan 19, 2023
erezrokah
reviewed
Jan 19, 2023
erezrokah
reviewed
Jan 19, 2023
yevgenypats
suggested changes
Jan 19, 2023
Contributor
yevgenypats
left a comment
yevgenypats
left a comment
There was a problem hiding this comment.
I think we should somehow export this default spec to filetypes library as it will be used in all "file" plugins
Co-authored-by: Erez Rokah <[email protected]> Co-authored-by: Herman Schaaf <[email protected]>
bbernays
commented
Jan 19, 2023
candiduslynx
approved these changes
Jan 19, 2023
Co-authored-by: Herman Schaaf <[email protected]>
bbernays
changed the title
bbernays
changed the title
hermanschaaf
approved these changes
Jan 27, 2023
Member
|
Docs wise (e.g. checking here) I don't think |
Collaborator
Author
Can we handle this in a follow up PR? |
disq
approved these changes
Jan 27, 2023
yevgenypats
approved these changes
Jan 27, 2023
This was referenced Jan 27, 2023
kodiakhq bot
pushed a commit
that referenced
this pull request
Jan 31, 2023
🤖 I have created a release *beep* *boop* --- ## [2.0.0](plugins-destination-gcs-v1.0.6...plugins-destination-gcs-v2.0.0) (2023-01-31) ### ⚠ BREAKING CHANGES * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ### Features * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ([d03819c](d03819c)) ### Bug Fixes * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.29.0 ([#7121](#7121)) ([b7441c9](b7441c9)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.30.0 ([#7222](#7222)) ([73ca21c](73ca21c)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.31.0 ([#7228](#7228)) ([36e8549](36e8549)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.32.0 ([#7334](#7334)) ([b684122](b684122)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot
pushed a commit
that referenced
this pull request
Jan 31, 2023
🤖 I have created a release *beep* *boop* --- ## [2.0.0](plugins-destination-s3-v1.1.1...plugins-destination-s3-v2.0.0) (2023-01-31) ### ⚠ BREAKING CHANGES * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) * **s3:** Clean inputs ([#7116](#7116)) ### Features * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ([d03819c](d03819c)) ### Bug Fixes * **deps:** Update module github.com/aws/aws-sdk-go-v2/feature/s3/manager to v1.11.49 ([#7309](#7309)) ([16da39d](16da39d)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.29.0 ([#7121](#7121)) ([b7441c9](b7441c9)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.30.0 ([#7222](#7222)) ([73ca21c](73ca21c)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.31.0 ([#7228](#7228)) ([36e8549](36e8549)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.32.0 ([#7334](#7334)) ([b684122](b684122)) * **s3:** Clean inputs ([#7116](#7116)) ([0132fb4](0132fb4)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot
pushed a commit
that referenced
this pull request
Jan 31, 2023
🤖 I have created a release *beep* *boop* --- ## [2.0.0](plugins-destination-azblob-v1.0.5...plugins-destination-azblob-v2.0.0) (2023-01-31) ### ⚠ BREAKING CHANGES * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ### Features * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ([d03819c](d03819c)) ### Bug Fixes * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.29.0 ([#7121](#7121)) ([b7441c9](b7441c9)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.30.0 ([#7222](#7222)) ([73ca21c](73ca21c)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.31.0 ([#7228](#7228)) ([36e8549](36e8549)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.32.0 ([#7334](#7334)) ([b684122](b684122)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot
pushed a commit
that referenced
this pull request
Jan 31, 2023
🤖 I have created a release *beep* *boop* --- ## [2.0.0](plugins-destination-kafka-v1.0.4...plugins-destination-kafka-v2.0.0) (2023-01-31) ### ⚠ BREAKING CHANGES * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ### Features * Add ability to override CSV File options (headers and delimiters) ([#6958](#6958)) ([d03819c](d03819c)) ### Bug Fixes * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.29.0 ([#7121](#7121)) ([b7441c9](b7441c9)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.30.0 ([#7222](#7222)) ([73ca21c](73ca21c)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.31.0 ([#7228](#7228)) ([36e8549](36e8549)) * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.32.0 ([#7334](#7334)) ([b684122](b684122)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Users can now use the following options to change the destination behavior:
skip_header: default isfalsewhich will mean the the first row will be the column headersdelimiter: default value is,but user can specify any single charactercloses #6795
closes #6797
closes #6796