Skip to content

Fix Scheduler address in standalone mode#8403

Closed
antontroshin wants to merge 9 commits into
dapr:release-1.15from
antontroshin:fix-scheduler-standalone-jobs-conn
Closed

Fix Scheduler address in standalone mode#8403
antontroshin wants to merge 9 commits into
dapr:release-1.15from
antontroshin:fix-scheduler-standalone-jobs-conn

Conversation

@antontroshin

Copy link
Copy Markdown
Contributor

Description

This PR attempts to fix the issue in Dapr runtime when used in standalone mode and it tries to connect to the scheduler but receives an error on the internal WatchHosts rpc call.
It returns internal Docker addresses that are inaccessible from the host machine.
In this case, will use supplied addresses via --scheduler-host-address argument, or if it is not supplied, use default local address :50006.

Issue reference

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Comment thread pkg/runtime/config.go Outdated
Fix for multiple scheduler addresses in standalone mode

Signed-off-by: Anton Troshin <[email protected]>
Comment thread pkg/runtime/config.go Outdated
@paulyuk

paulyuk commented Jan 23, 2025

Copy link
Copy Markdown
Contributor

Hey @antontroshin will this fix #8412 or is that likely a different issue?

@antontroshin

Copy link
Copy Markdown
Contributor Author

The latest CLI RC2 has this issue fixed for quickstarts dapr/cli#1475
The issue may be experienced on local machine when the version does not pass the "edge"/version check and parameter is being removed.
Added this PR (dapr/cli#1479), to add "dev" version tag to this check.

@antontroshin
antontroshin marked this pull request as ready for review January 24, 2025 05:51
@antontroshin
antontroshin requested review from a team as code owners January 24, 2025 05:51
Comment thread pkg/runtime/scheduler/scheduler.go Outdated
Comment on lines +264 to +266
if len(s.addresses) > 0 && s.mode == modes.StandaloneMode {
addresses = s.addresses
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't do this- this is going to fail in standalone mode on an existing deployment when the scheduler replicas are scaled up (or down). This is one of the main reasons for this API existing in the first place. Not least, daprd side hacks like these are super fragile and can back us into a corner long term.

If scheduler is listening on localhost and this is inaccessible to the host machine, how is scheduler reachable at all? If it's listening on another interface, then that IP too should be reachable from the host machine and the existing logic in Scheduler to use the right address to broadcast should also work.

If we must must do something here, I think it best to add a --override-broadcast-address CLI flag on the Scheduler side.

@antontroshin antontroshin linked an issue Jan 24, 2025 that may be closed by this pull request
yaron2 and others added 3 commits January 24, 2025 14:04
Signed-off-by: Anton Troshin <[email protected]>
@antontroshin

Copy link
Copy Markdown
Contributor Author

Superseded by #8416

@antontroshin
antontroshin deleted the fix-scheduler-standalone-jobs-conn branch January 27, 2025 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants