-
Notifications
You must be signed in to change notification settings - Fork 641
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Search before asking
- I had searched in the issues and found no similar issues.
Enhancement Request
In the various implementations of Source, there are often some common configurations scattered across different places, such as the capacity of the storage queue, the number of elements to poll each time, etc. Additionally, in some source implementations, the poll method can block for up to several seconds in the worst-case scenario, reaching maxBatchSize * pollTimeout. Therefore, it is necessary to define common parameters that support configuration and also limit the maximum waiting time for poll.
Describe the solution you'd like
- Added a
PollConfigconfiguration class that includes parameters forcapacity,maxBatchSize, andmaxWaitTime, and incorporated it as a variable inSourceConfigto support user configuration with default values. - Replaced the fixed constants for
capacity,maxBatchSize, andmaxWaitTimein the original source implementations with the corresponding variables fromPollConfig. - Optimized some
pollmethods to ensure the maximum waiting time does not exceedmaxWaitTime, preventing blocking for several seconds.
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct *
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request