Skip to content

Conversation

@erezrokah
Copy link
Member

Summary

Fixes #12674 + destinations options + some indentation fixes.
I chose to keep the options on the same page and only move them to the bottom.
This way links to the options don't break.

We should also add the specific options to each plugin, but that's more time consuming

@cq-bot cq-bot added the website label Aug 1, 2023
name: firestore
path: cloudquery/firestore
version: "VERSION_SOURCE_FIRESTORE"
concurrency: 10000
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not supported in Firestore (like most other DB sources)

name: "mysql"
path: "cloudquery/mysql"
version: "VERSION_SOURCE_MYSQL"
concurrency: 10000
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not supported in MySQL (like most other DB sources)

name: "oracledb"
path: "cloudquery/oracledb"
version: "VERSION_SOURCE_ORACLEDB"
concurrency: 10000
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supported in OracleDB source but the default of 100 should be good for now unless someone reports it

@erezrokah erezrokah added the automerge Automatically merge once required checks pass label Aug 1, 2023
@erezrokah erezrokah requested a review from hermanschaaf August 1, 2023 15:26
}

func (s *Spec) SetDefaults() {
if s.Concurrency == 0 {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default was missing from this plugin


const (
defaultConccurency = 10000
defaultConcurrency = 10000
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo


func (s *Spec) SetDefaults() {
if s.Concurrency == 0 {
s.Concurrency = 10000
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added missing default

@erezrokah
Copy link
Member Author

erezrokah commented Aug 2, 2023

OK so I went through each plugin's implementation and updated the docs for each plugin with the relevant concurrency, scheduler (only GCP) and backend_options options and defaults.

I also removed the concurrency: 10000 we had in many configurations as now each plugin has a sensible default that should work without needing to override it.

@kodiakhq kodiakhq bot merged commit 4108210 into cloudquery:main Aug 2, 2023
@erezrokah erezrokah deleted the docs/source_plugin_spec branch August 2, 2023 14:27
kodiakhq bot pushed a commit that referenced this pull request Aug 4, 2023
🤖 I have created a release *beep* *boop*
---


## [3.1.3](plugins-source-datadog-v3.1.2...plugins-source-datadog-v3.1.3) (2023-08-04)


### Bug Fixes

* **deps:** Update github.com/apache/arrow/go/v13 digest to 112f949 ([#12659](#12659)) ([48d73a9](48d73a9))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 3452eb0 ([#12595](#12595)) ([c1c0949](c1c0949))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.4 ([#12718](#12718)) ([f059a15](f059a15))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.5 ([#12731](#12731)) ([d267239](d267239))
* Return a datadog client instance instead of nil ([#12766](#12766)) ([9d6c354](9d6c354))
* **website:** Move deprecated top level spec options to plugin level ([#12686](#12686)) ([4108210](4108210))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot pushed a commit that referenced this pull request Aug 8, 2023
🤖 I have created a release *beep* *boop*
---


## [5.0.3](plugins-source-cloudflare-v5.0.2...plugins-source-cloudflare-v5.0.3) (2023-08-08)


### Bug Fixes

* **deps:** Update github.com/apache/arrow/go/v13 digest to 112f949 ([#12659](#12659)) ([48d73a9](48d73a9))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 3452eb0 ([#12595](#12595)) ([c1c0949](c1c0949))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to f53878d ([#12778](#12778)) ([6f5d58e](6f5d58e))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.4 ([#12718](#12718)) ([f059a15](f059a15))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.5 ([#12731](#12731)) ([d267239](d267239))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.6 ([#12799](#12799)) ([fb0e0d7](fb0e0d7))
* **website:** Move deprecated top level spec options to plugin level ([#12686](#12686)) ([4108210](4108210))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
kodiakhq bot pushed a commit that referenced this pull request Aug 8, 2023
🤖 I have created a release *beep* *boop*
---


## [4.0.3](plugins-source-alicloud-v4.0.2...plugins-source-alicloud-v4.0.3) (2023-08-08)


### Bug Fixes

* **deps:** Update github.com/apache/arrow/go/v13 digest to 112f949 ([#12659](#12659)) ([48d73a9](48d73a9))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to 3452eb0 ([#12595](#12595)) ([c1c0949](c1c0949))
* **deps:** Update github.com/cloudquery/arrow/go/v13 digest to f53878d ([#12778](#12778)) ([6f5d58e](6f5d58e))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.4 ([#12718](#12718)) ([f059a15](f059a15))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.5 ([#12731](#12731)) ([d267239](d267239))
* **deps:** Update module github.com/cloudquery/plugin-sdk/v4 to v4.2.6 ([#12799](#12799)) ([fb0e0d7](fb0e0d7))
* **website:** Move deprecated top level spec options to plugin level ([#12686](#12686)) ([4108210](4108210))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge Automatically merge once required checks pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: Update docs for backend, backend_spec , scheduler and concurrency spec options

3 participants