-
Notifications
You must be signed in to change notification settings - Fork 547
feat(sqlite): Migrate to apache arrow type system #9751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
erezrokah
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did a quick review of the code to make sure I get a general idea.
If you can update the go.mod to update to commit I can also test (or tell me which versions of SDK/Arrow to use).
71eed8d to
695d677
Compare
c71116f to
1717ff0
Compare
Thanks @erezrokah go.mod should be up to date now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested this on multiple sources (AWS, GCP and Azure) and upgrading between major versions using forced migration mode. Looks great 🚀
🤖 I have created a release *beep* *boop* --- ## [2.0.0](plugins-destination-sqlite-v1.3.6...plugins-destination-sqlite-v2.0.0) (2023-04-17) ### ⚠ BREAKING CHANGES * This release introduces an internal change to our type system to use [Apache Arrow](https://arrow.apache.org/). This should not have any visible breaking changes, however due to the size of the change we are introducing it under a major version bump to communicate that it might have some bugs that we weren't able to catch during our internal tests. If you encounter an issue during the upgrade, please submit a [bug report](https://github.com/cloudquery/cloudquery/issues/new/choose). ### Features * **sqlite:** Migrate to apache arrow type system ([#9751](#9751)) ([549a38e](549a38e)) * Update to use arrow type system ([a55e0fb](a55e0fb)) ### Bug Fixes * **deps:** Update module github.com/cloudquery/plugin-sdk to v1.45.0 ([#9863](#9863)) ([2799d62](2799d62)) * **sqlite:** Make arrow work with older sources ([#10141](#10141)) ([eef77bb](eef77bb)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This still wait for another sdkv2 release for tests to pass but should be ready for initial review.