Skip to content

Add queueEndpoint to docs for Azure Storage Queues binding#2424

Merged
yaron2 merged 6 commits into
dapr:v1.8from
corechain-technologies:azure-storage-queues-custom-endpoint
May 13, 2022
Merged

Add queueEndpoint to docs for Azure Storage Queues binding#2424
yaron2 merged 6 commits into
dapr:v1.8from
corechain-technologies:azure-storage-queues-custom-endpoint

Conversation

@rzyns

@rzyns rzyns commented May 5, 2022

Copy link
Copy Markdown
Contributor

Thank you for helping make the Dapr documentation better!

Please follow this checklist before submitting:

  • Commits are signed with Developer Certificate of Origin (DCO - learn more)
  • Read the contribution guide
  • Commands include options for Linux, MacOS, and Windows within codetabs
  • New file and folder names are globally unique
  • Page references use shortcodes instead of markdown or URL links
  • Images use HTML style and have alternative text
  • Places where multiple code/command options are given have codetabs

Description

Add example for the queueEndpoint binding introduced by dapr/components-contrib#1692

Issue reference

dapr/components-contrib#1692

@msfussell

msfussell commented May 6, 2022

Copy link
Copy Markdown
Member

@rzyns - Please update the Spec metadata fields table, which provides information on what this does and how to use this. Also can you target the v1.8 release branch as this will not be available until that release.

@msfussell
msfussell requested a review from berndverst May 6, 2022 00:13
@msfussell msfussell added this to the 1.8 milestone May 6, 2022
@rzyns
rzyns changed the base branch from v1.7 to v1.8 May 6, 2022 00:21
@rzyns
rzyns marked this pull request as draft May 6, 2022 00:21
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from 6d1777a to 80ca7e1 Compare May 6, 2022 01:10
@rzyns
rzyns marked this pull request as ready for review May 6, 2022 01:10
@rzyns
rzyns marked this pull request as draft May 6, 2022 01:10
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from 80ca7e1 to 79bbfd4 Compare May 6, 2022 01:14
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from 79bbfd4 to f3c4248 Compare May 6, 2022 01:17
Signed-off-by: Janusz Dziurzynski <[email protected]>
@rzyns
rzyns marked this pull request as ready for review May 6, 2022 01:39
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from c2eeb53 to 9485bbc Compare May 6, 2022 14:27
…indings/storagequeues.md

Co-authored-by: Mark Fussell <[email protected]>
Signed-off-by: Janusz Dziurzynski <[email protected]>
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from 9485bbc to 0439f7e Compare May 6, 2022 14:27
| queue | Y | Input/Output | The name of the Azure Storage queue | `"myqueue"` |
| ttlInSeconds | N | Output | Parameter to set the default message time to live. If this parameter is omitted, messages will expire after 10 minutes. See [also](#specifying-a-ttl-per-message) | `"60"` |
| decodeBase64 | N | Output | Configuration to decode base64 file content before saving to Blob Storage. (In case of saving a file with binary content). `true` is the only allowed positive value. Other positive variations like `"True", "1"` are not acceptable. Defaults to `false` | `true`, `false` |
| queueEndpoint | N | Input/Output | Optional custom endpoint URL. This is useful, for example, with "[production-style URLs](https://github.com/Azure/azurite#production-style-url)" when using the [Storage emulator](https://github.com/Azure/azurite) | `"https://accountName.queue.example.com:10001"` |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this actually useful without the storage emulator? I couldn't find any example. If it is limited to Azurite / emulators I think we should make it more clear like.. Optional. Only applies to use with Storage Emulator.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rzyns - Can you reply to this question from @berndverst?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry @berndverst, I didn't notice you had posted a comment on this issue--I was mostly watching for activity on the main PR. Anyway, this isn't only useful for the storage emulator, it also be useful when you have a storage endpoint mapped to a custom domain. I haven't tried that with storage queues yet and so I haven't explicitly tested it, but I wouldn't say it's "only useful with the storage emulator." It's also useful for integration testing, or any scenario in which you need to direct the API traffic to a specific endpoint.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that you add other cases where this queueEndpoint can be used, as you described.

Also ideally this property should IMO be called "queueEndpointUrl" is this is a URL.

@rzyns rzyns May 10, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree about the field name, @msfussell, and I've committed the change. I'm reluctant to add other use cases to the documentation, though, since I've only explicitly tested the storage emulator case. If I do get a chance to actually test out the custom domain case later, I'd be happy to submit another PR

@msfussell msfussell May 10, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rzyns - I would still mention these as scenarios, and feel to add a caveat that these have not been tested. That way, you may get someone else to try this out and give a response.

rzyns added a commit to corechain-technologies/dapr-components-contrib that referenced this pull request May 10, 2022
Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <[email protected]>
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from a43731d to 7f3c97d Compare May 10, 2022 17:18
rzyns added 2 commits May 10, 2022 13:19
Signed-off-by: Janusz Dziurzynski <[email protected]>
Signed-off-by: Janusz Dziurzynski <[email protected]>
@rzyns
rzyns force-pushed the azure-storage-queues-custom-endpoint branch from 7f3c97d to ba3a8d1 Compare May 10, 2022 17:32
rzyns added a commit to corechain-technologies/dapr-components-contrib that referenced this pull request May 10, 2022
Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <[email protected]>

@msfussell msfussell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonderful contribution! LGTM.
@rzyns - Looks like this branch is out of date. Can you address this?

yaron2 pushed a commit to dapr/components-contrib that referenced this pull request May 10, 2022
* Support custom queueEndpoint in Azure Storage Queues

Signed-off-by: Janusz Dziurzynski <[email protected]>

* run gofmt

Signed-off-by: Janusz Dziurzynski <[email protected]>

* Add "Url" to JSON field name for clarity

Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <[email protected]>
@msfussell

Copy link
Copy Markdown
Member

@rzyns - Ping on the comment above^^

@rzyns

rzyns commented May 13, 2022

Copy link
Copy Markdown
Contributor Author

Thanks, @msfussell! Sorry for the delay, should be up-to-date now. LMK if you want me to rebase instead

ewassef pushed a commit to ewassef/components-contrib that referenced this pull request May 13, 2022
* Support custom queueEndpoint in Azure Storage Queues

Signed-off-by: Janusz Dziurzynski <[email protected]>

* run gofmt

Signed-off-by: Janusz Dziurzynski <[email protected]>

* Add "Url" to JSON field name for clarity

Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <[email protected]>
Signed-off-by: Eddie <[email protected]>
@yaron2
yaron2 merged commit 7f8532b into dapr:v1.8 May 13, 2022
yaron2 added a commit to dapr/components-contrib that referenced this pull request May 13, 2022
* Update readme of bindings (#1690)

Signed-off-by: pigletfly <[email protected]>

Co-authored-by: Looong Dai <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Fixing includedHeaders problem with spaces (#1610)

Signed-off-by: Ben Kotvis <[email protected]>

Co-authored-by: Bernd Verst <[email protected]>
Co-authored-by: Looong Dai <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Simplify vault token read (#1560)

* Simplify vault token get

Signed-off-by: zhangchao <[email protected]>

* fix lint

Signed-off-by: zhangchao <[email protected]>

* update tests

Signed-off-by: zhangchao <[email protected]>

Co-authored-by: Looong Dai <[email protected]>
Co-authored-by: Yaron Schneider <[email protected]>
Signed-off-by: Eddie <[email protected]>

* GH-1609 : Fix for MongoDB Atlas conn strings

Added recommended fix in the issue

Signed-off-by: Eddie <[email protected]>

* updating the comment based on PR feedback

Signed-off-by: Eddie <[email protected]>

* Initial Certification test for eventhubs binding [incomplete] (#1670)

* certification test for eventhubs binding

Signed-off-by: tanvigour <[email protected]>

* modified go.mod and go.sum

Signed-off-by: tanvigour <[email protected]>

* Add connection string testing

Signed-off-by: tanvigour <[email protected]>

* iothub testing

Signed-off-by: tanvigour <[email protected]>

* address feedback and run test

Signed-off-by: tanvigour <[email protected]>

* Install Azure CLI IOT hub extension

Signed-off-by: Bernd Verst <[email protected]>

* make modtidy-all

Signed-off-by: Bernd Verst <[email protected]>

* covering all eventhubs test cases

Signed-off-by: tanvigour <[email protected]>

* dependency changes after go modtidy-all

Signed-off-by: tanvigour <[email protected]>

Co-authored-by: Bernd Verst <[email protected]>
Co-authored-by: Yaron Schneider <[email protected]>
Co-authored-by: Looong Dai <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Use revive instead of golint (#1685)

Signed-off-by: pigletfly <[email protected]>

Co-authored-by: Yaron Schneider <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Updated to Go 1.18 (#1697)

* Updated to Go 1.18

Signed-off-by: Alessandro (Ale) Segala <[email protected]>

* Added go.work file
With Go 1.18, this allows gopls (the Go language server used for example in VS Code) to work inside test apps too.
See: https://go.dev/doc/tutorial/workspaces

Signed-off-by: ItalyPaleAle <[email protected]>

Signed-off-by: ItalyPaleAle <[email protected]>

* Removed go.work

Signed-off-by: ItalyPaleAle <[email protected]>

* 💄

Signed-off-by: ItalyPaleAle <[email protected]>

Co-authored-by: Bernd Verst <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Add metadata property to configure Batching in Pulsar (#1707)

* Add metadata property to configure BatchingMaxSize&batchingMaxMessages in Pulsar
Signed-off-by: saberwang <[email protected]>

* sort field
Signed-off-by: saberwang <[email protected]>

* [pubsub]fix unit test bug
Signed-off-by: saberwang <[email protected]>

* remove unrelated changes
Signed-off-by: saberwang <[email protected]>

* Delete hard coded Metadata
Signed-off-by: saberwang <[email protected]>

* remove  .history

Signed-off-by: saberwang <[email protected]>

* restore .gitignore

Signed-off-by: saberwang <[email protected]>

* Hard coding default values and adding 'BatchingMaxPublishDelay' metadata

Signed-off-by: saberwang <[email protected]>

* fix code format

Signed-off-by: saberwang <[email protected]>

* formatting code

Signed-off-by: saberwang <[email protected]>

Co-authored-by: Looong Dai <[email protected]>
Co-authored-by: Bernd Verst <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Fix 4529: Ignore Subscribe/Get wrong redis configuration type keys. (#1693)

* fix: 4529

Signed-off-by: LaurenceLiZhixin <[email protected]>

* Fix: add test does not throw error for wrong type during get all test case of redis configuration

Signed-off-by: LaurenceLiZhixin <[email protected]>

Co-authored-by: Yaron Schneider <[email protected]>
Co-authored-by: Ian Luo <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Signed-off-by: Eddie Wassef <[email protected]>

Fixing leading newline in linter error

Signed-off-by: Eddie <[email protected]>

* Implment a secret store based on Huawei CSMS (#1710)

Signed-off-by: Chen Cong <[email protected]>

Co-authored-by: Chen Cong <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Add yet another missing secret to eventhub binding cert test (#1713)

Signed-off-by: Bernd Verst <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Support custom queueEndpoint in Azure Storage Queues (#1692)

* Support custom queueEndpoint in Azure Storage Queues

Signed-off-by: Janusz Dziurzynski <[email protected]>

* run gofmt

Signed-off-by: Janusz Dziurzynski <[email protected]>

* Add "Url" to JSON field name for clarity

Suggested by @msfussell in dapr/docs#2424

Signed-off-by: Janusz Dziurzynski <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Refactory kafka binding to reuse the kafka common code extracting from kafka pubsub component (#1696)

* refactory kafka pubsub code to extract common kafka code for reuse

Signed-off-by: Sky Ao <[email protected]>

* fix lint;add unit test for subscribeAdapter

Signed-off-by: Sky Ao <[email protected]>

* move topics filed from internal kafak struct to pubsub kafka struct, since in input binding the topics will confiured in metadata

Signed-off-by: Sky Ao <[email protected]>

* reuse internal  kafka code for bindings

Signed-off-by: Sky Ao <[email protected]>

* add redis standalone_test back which is delete by mistaken

Signed-off-by: Sky Ao <[email protected]>

* small code improvement to trigger test

Signed-off-by: Sky Ao <[email protected]>

* add license headers

Signed-off-by: Sky Ao <[email protected]>

* try to set disbaleTls to true to verify the kafka connection fail

Signed-off-by: Sky Ao <[email protected]>

* don't enable consum retry in kafka binding component;if authenticaion is disabled, need not set TLSDisable at the same time;

Signed-off-by: Sky Ao <[email protected]>

* fix lint

Signed-off-by: Sky Ao <[email protected]>

Co-authored-by: Loong Dai <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Add topic metadata for mqtt input binding and support user defined topic for mqtt output binding (#1674)

* feat(bindings/mqtt): add data incoming topic to metadata

Signed-off-by: lotuc <[email protected]>

* feat(bindings/mqtt): support user defined topic on create action

Signed-off-by: lotuc <[email protected]>

* chore(bindings/mqtt): add integration test and topic response check test

Signed-off-by: lotuc <[email protected]>

* fix(bindings/mqtt): ignore misspell linting error for word mosquitto

Signed-off-by: lotuc <[email protected]>

Co-authored-by: Yaron Schneider <[email protected]>
Signed-off-by: Eddie <[email protected]>

* Expire -> ExpiryInSeconds (#1721)

Signed-off-by: seeflood <[email protected]>
Signed-off-by: Eddie <[email protected]>

* running gofmt -s -w state/mongodb/mongodb.go

Signed-off-by: Eddie <[email protected]>

* Update mongodb.go

Co-authored-by: Wang Bing <[email protected]>
Co-authored-by: Looong Dai <[email protected]>
Co-authored-by: Ben Kotvis <[email protected]>
Co-authored-by: Bernd Verst <[email protected]>
Co-authored-by: Taction <[email protected]>
Co-authored-by: Yaron Schneider <[email protected]>
Co-authored-by: tanvigour <[email protected]>
Co-authored-by: Alessandro (Ale) Segala <[email protected]>
Co-authored-by: saber-wang <[email protected]>
Co-authored-by: Laurence <[email protected]>
Co-authored-by: Ian Luo <[email protected]>
Co-authored-by: Chock Chen <[email protected]>
Co-authored-by: Chen Cong <[email protected]>
Co-authored-by: Janusz Dziurzynski <[email protected]>
Co-authored-by: Sky Ao <[email protected]>
Co-authored-by: lotuc <[email protected]>
Co-authored-by: seeflood <[email protected]>
@rzyns
rzyns deleted the azure-storage-queues-custom-endpoint branch May 19, 2022 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants