Emit event when a datasource has been dropped from the coordinator#18497
Conversation
kfaraz
left a comment
kfaraz
left a comment
There was a problem hiding this comment.
Since we already have this PR, do we still need the one disabling the test?
|
@kfaraz yeah let me do that. |
| .hasDimension(DruidMetrics.DATASOURCE, dataSource), | ||
| agg -> agg.hasSumAtLeast(numSegments) | ||
| ); | ||
| broker.latchableEmitter().waitForEvent( |
There was a problem hiding this comment.
Maybe we should we look for aggregate where the sum is equal to numSegments.
After this change, do we even need to wait to wait for the load success.
@uds5501 , could you try to push a change with the coordinator part commented out?
If not needed, we can remove that check completely.
There was a problem hiding this comment.
Tested locally, the tests seems to be working fine without the coordinator wait, removed in the latest comment.
Also updated the EmbeddedBroker as part of the numSegments change.
kfaraz
left a comment
kfaraz
left a comment
There was a problem hiding this comment.
Looks good, left some minor suggestions.
This reverts commit 805e255.
|
One input source IT was running. The patch does not change anything there hence merged the PR. |
…pache#18497) * Emit event when a datasource has been dropped from the coordinator * Fix checkstyle * Accommodate review comments * Missed pusshing metric * Reenable flaky test * Wait for broker event for coordinator pause tests * Fix auto compact test issues * Make checkstyle happy. * Remove coordinator load checks and accommodate review suggestions. * Disable Centralized Schema publish failures * Disable Centralized Schema Metadata query tests too * Fix latachable emitter issue * Disabling few more tests due to timeouts * Revert "Disabling few more tests due to timeouts" This reverts commit 805e255. * Disabling Auto Compact Tests
4 participants
Follow up PR for #18495
It introduces a new event
segment/schemaCache/datasource/dropped/countthat is emitted when the data source is dropped from the coordinator and the cache refresh in broker acknowledges it.