Skip to content

feat: Add unmanaged sources #677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
kodiakhq merged 4 commits into from
Feb 13, 2023
Merged

feat: Add unmanaged sources #677

kodiakhq merged 4 commits into from
Feb 13, 2023

Conversation

@yevgenypats
Copy link
Contributor

@yevgenypats yevgenypats commented Feb 12, 2023

Used in this postgres source - cloudquery/cloudquery#7835

@yevgenypats
feat: Add unmanaged sources
d6787f5 
Usually used in database sources
@github-actions github-actions bot added the feat label Feb 12, 2023
@github-actions
Copy link

github-actions bot commented Feb 12, 2023 •
edited
Loading

⏱️ Benchmark results

Comparing with 8e8f5fe

  • DefaultConcurrencyDFS-2 resources/s: 11,120 ⬆️ 4.20% increase vs. 8e8f5fe
  • DefaultConcurrencyRoundRobin-2 resources/s: 12,155 ⬆️ 2.68% increase vs. 8e8f5fe
  • Glob-2 ns/op: 192.4 ⬇️ 34.62% decrease vs. 8e8f5fe
  • TablesWithChildrenDFS-2 resources/s: 28,931 ⬆️ 8.43% increase vs. 8e8f5fe
  • TablesWithChildrenRoundRobin-2 resources/s: 27,120 ⬆️ 10.21% increase vs. 8e8f5fe
  • TablesWithRateLimitingDFS-2 resources/s: 28.35 ⬇️ 0.28% decrease vs. 8e8f5fe
  • TablesWithRateLimitingRoundRobin-2 resources/s: 824.2 ⬇️ 6.94% decrease vs. 8e8f5fe
  • BufferedScanner-2 ns/op: 9.382 ⬇️ 20.98% decrease vs. 8e8f5fe
  • LogReader-2 ns/op: 30.99 ⬇️ 18.62% decrease vs. 8e8f5fe

@codecov-commenter
Copy link

codecov-commenter commented Feb 12, 2023 •
edited
Loading

Codecov Report

Base: 47.23% // Head: 46.55% // Decreases project coverage by -0.69% ⚠️

Coverage data is based on head (e1cf405) compared to base (879f843).
Patch has no changes to coverable lines.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #677      +/-   ##
==========================================
- Coverage   47.23%   46.55%   -0.69%     
==========================================
  Files          70       55      -15     
  Lines        6815     5411    -1404     
==========================================
- Hits         3219     2519     -700     
+ Misses       3146     2531     -615     
+ Partials      450      361      -89
Impacted Files Coverage Δ
schema/meta.go 10.00% <ø> (ø)
plugins/source/options.go
plugins/source/plugin.go
serve/enum.go
plugins/source/scheduler.go
plugins/source/metrics.go
serve/source.go
plugins/source/scheduler_dfs.go
plugins/source/scheduler_round_robin.go
plugins/source/docs.go
... and 7 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

erezrokah
erezrokah approved these changes Feb 12, 2023
View reviewed changes
@yevgenypats yevgenypats marked this pull request as ready for review February 13, 2023 08:35
@kodiakhq kodiakhq bot merged commit f3e2b1d into main Feb 13, 2023
@kodiakhq kodiakhq bot deleted the feat/unmanaged_source branch February 13, 2023 08:45
@cq-bot cq-bot mentioned this pull request Feb 13, 2023
Merged
kodiakhq bot pushed a commit that referenced this pull request Feb 13, 2023
@cq-bot
chore(main): Release v1.37.0 (#684)
a1409ac 
🤖 I have created a release *beep* *boop*
---


## [1.37.0](v1.36.3...v1.37.0) (2023-02-13)


### Features

* Add unmanaged sources ([#677](#677)) ([f3e2b1d](f3e2b1d))


### Bug Fixes

* Fix race in dest testing try 3 ([#683](#683)) ([8e8f5fe](8e8f5fe))
* Make sure _cq_id unique across all dest plugins ([#685](#685)) ([a9a1173](a9a1173))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
@erezrokah erezrokah mentioned this pull request Feb 13, 2023
Merged
1 task
kodiakhq bot pushed a commit to cloudquery/cloudquery that referenced this pull request Feb 13, 2023
@cq-bot
fix(deps): Update module github.com/cloudquery/plugin-sdk to v1.37.0 (#…
dc9cffb 
…7933)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [github.com/cloudquery/plugin-sdk](https://togithub.com/cloudquery/plugin-sdk) | require | patch | `v1.36.1` -> `v1.37.0` |
| [github.com/cloudquery/plugin-sdk](https://togithub.com/cloudquery/plugin-sdk) | require | patch | `v1.36.0` -> `v1.37.0` |

---

### Release Notes

<details>
<summary>cloudquery/plugin-sdk</summary>

### [`v1.37.0`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v1.37.0)

[Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v1.36.3...v1.37.0)

##### Features

* Add unmanaged sources ([#677](cloudquery/plugin-sdk#677)) ([f3e2b1d](cloudquery/plugin-sdk@f3e2b1d))


##### Bug Fixes

* Fix race in dest testing try 3 ([#683](cloudquery/plugin-sdk#683)) ([8e8f5fe](cloudquery/plugin-sdk@8e8f5fe))
* Make sure _cq_id unique across all dest plugins ([#685](cloudquery/plugin-sdk#685)) ([a9a1173](cloudquery/plugin-sdk@a9a1173))

## [1.36.3](cloudquery/plugin-sdk@v1.36.2...v1.36.3) (2023-02-12)

### [`v1.36.3`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v1.36.3)

[Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v1.36.2...v1.36.3)

##### Bug Fixes

-   Take2 of fixing race in destination testing ([#&#8203;680](https://togithub.com/cloudquery/plugin-sdk/issues/680)) ([77b74b2](https://togithub.com/cloudquery/plugin-sdk/commit/77b74b2cd4c28ee5f570b008105a41d0b7e8afc8))

### [`v1.36.2`](https://togithub.com/cloudquery/plugin-sdk/releases/tag/v1.36.2)

[Compare Source](https://togithub.com/cloudquery/plugin-sdk/compare/v1.36.1...v1.36.2)

##### Bug Fixes

-   Potential database lock/race in destination testing ([#&#8203;678](https://togithub.com/cloudquery/plugin-sdk/issues/678)) ([50e683e](https://togithub.com/cloudquery/plugin-sdk/commit/50e683e7f6dfd38a25eb512c1e2417798fa832f7))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xMDkuMSIsInVwZGF0ZWRJblZlciI6IjM0LjEwOS4xIn0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erezrokah erezrokah erezrokah approved these changes

@hermanschaaf hermanschaaf Awaiting requested review from hermanschaaf

Assignees

No one assigned

Labels

automerge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yevgenypats @codecov-commenter @erezrokah