@alenkacz, thanks for offering to help with this! Per @olix0r's comment in #1640, it would be great for the --wait flag to specify a duration to wait fo the checks to succeed, and default the flag to 300s.

From an implementation standpoint, I think we should modify the flag type in cmd/cli, and then modify the healthcheck.HealthCheckOptions struct to support a RetryDeadline field, instead of the existing ShouldRetry field. We can set the value of RetryDeadline using the value of the --wait flag. I'm picturing something like:

@@ -88,7 +89,7 @@ func configureAndRunChecks(options *checkOptions) {
                KubeConfig:                     kubeconfigPath,
                APIAddr:                        apiAddr,
                VersionOverride:                options.versionOverride,
-               ShouldRetry:                    options.wait,
+               RetryDeadline:                  time.Now().Add(options.wait),
                ShouldCheckKubeVersion:         true,
                ShouldCheckControlPlaneVersion: !(options.preInstallOnly || options.dataPlaneOnly),
                ShouldCheckDataPlaneVersion:    options.dataPlaneOnly,

When performing the checks in healthcheck.go, instead of checking if maxRetries has been exceeded, we can instead check if the current time is before the deadline. Something like:

@@ -378,11 +377,6 @@ func (hc *HealthChecker) RunChecks(observer checkObserver) bool {
 }
 
 func (hc *HealthChecker) runCheck(c *checker, observer checkObserver) bool {
-       var retries int
-       if c.retry {
-               retries = maxRetries
-       }
-
        for {
                err := c.check()
                checkResult := &CheckResult{
@@ -391,8 +385,7 @@ func (hc *HealthChecker) runCheck(c *checker, observer checkObserver) bool {
                        Err:         err,
                }
 
-               if err != nil && retries > 0 {
-                       retries--
+               if err != nil && c.retry && time.Now().Before(hc.RetryDeadline) {
                        checkResult.Retry = true
                        observer(checkResult)
                        time.Sleep(retryWindow)

Note that in order to do this, we'd need to explicitly set retry: true on all retryable checks, instead of retry: hc.ShouldRetry, as it's currently written.

Increase --wait timeout #1586

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Increase `--wait` timeout #1586