Skip to content

Cleaner callbacks for polling status#258

Merged
nfx merged 4 commits intomainfrom
fix/251
Dec 23, 2022
Merged

Cleaner callbacks for polling status#258
nfx merged 4 commits intomainfrom
fix/251

Conversation

@nfx
Copy link
Copy Markdown
Contributor

@nfx nfx commented Dec 19, 2022

This PR ensures that i.Info == nil is a clear indicator for a long-running operation callback that it's run in the polling mode. For convenience, retries.OnPoll is added as well:

clstr, err := w.Clusters.CreateAndWait(ctx, clusters.CreateCluster{
		ClusterName:            clusterName,
		SparkVersion:           latest,
		InstancePoolId:         GetEnvOrSkipTest(t, "TEST_INSTANCE_POOL_ID"),
		AutoterminationMinutes: 15,
		NumWorkers:             1,
	}, retries.Timeout[clusters.ClusterInfo](20*time.Minute),
		retries.OnPoll(func(i *clusters.ClusterInfo) {
			t.Logf("cluster is %s", i.State)
		}))

Fixes #251

@nfx nfx requested a review from a team December 19, 2022 12:04
Copy link
Copy Markdown
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, the PR summary didn't make sense to me until I read the PR diff.

"enforces" made me look for an assert, but this changes the type to a pointer so that a nil value can have special meaning.

What is the motivation for this change? Nevermind, I now see the linked issue.

@nfx nfx enabled auto-merge (squash) December 19, 2022 17:18
Copy link
Copy Markdown
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you take a look at the comments?

nfx added 3 commits December 23, 2022 16:37
This PR enforces `*retries.Info[T] { Info *T }`, so that `i.Info == nil` is a clear indicator to a callback that it's run in the polling mode.

For convenience, `retries.OnPoll` is added as well:

```go
clstr, err := w.Clusters.CreateAndWait(ctx, clusters.CreateCluster{
		ClusterName:            clusterName,
		SparkVersion:           latest,
		InstancePoolId:         GetEnvOrSkipTest(t, "TEST_INSTANCE_POOL_ID"),
		AutoterminationMinutes: 15,
		NumWorkers:             1,
	}, retries.Timeout[clusters.ClusterInfo](20*time.Minute),
		retries.OnPoll(func(i *clusters.ClusterInfo) {
			t.Logf("cluster is %s", i.State)
		}))
```

Fixes #251
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

Base: 42.53% // Head: 42.53% // No change to project coverage 👍

Coverage data is based on head (b500164) compared to base (c5a4f52).
Patch coverage: 0.00% of modified lines in pull request are covered.

❗ Current head b500164 differs from pull request most recent head 37a4a8f. Consider uploading reports for the commit 37a4a8f to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #258   +/-   ##
=======================================
  Coverage   42.53%   42.53%           
=======================================
  Files          42       42           
  Lines        2464     2464           
=======================================
  Hits         1048     1048           
  Misses       1336     1336           
  Partials       80       80           
Impacted Files Coverage Δ
service/clusters/api.go 0.00% <0.00%> (ø)
service/commands/api.go 42.85% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@nfx nfx requested a review from pietern December 23, 2022 15:42
@nfx nfx disabled auto-merge December 23, 2022 15:42
@nfx nfx merged commit d8cd795 into main Dec 23, 2022
@nfx nfx deleted the fix/251 branch December 23, 2022 15:43
@nfx nfx mentioned this pull request Dec 23, 2022
nfx added a commit that referenced this pull request Dec 23, 2022
# Version changelog

## 0.2.0

* Added `DATABRICKS_AUTH_TYPE` environment variable
([#248](#248)).
* Added Policy Families API
([#263](#263)).
* Added experimental `ErrCannotConfigureAuth` and `ErrNotAccountClient`
([#237](#237),
[#238](#238)).
* Added DBFS file handle that supports both reading and writing
([#261](#261)).
* Added `io.WriterTo` for DBFS file reader
([#249](#249)).
* Added `pflag.Value` interfaces for enums
([#234](#234)).
* Added support for adding custom HTTP visitors per request
([#230](#230)).
* Added support for raw body as byte slice if requested
([#247](#247)).
* Improved callbacks for polling the status of long-running operations
([#258](#258)).
* Improved rendering of HTTP links in godoc
([#229](#229)).
* Updated field types in the Jobs API from spec
([#259](#259)).
* Multiple OpenAPI consistency passes
([#254](#254),
[#241](#241),
[#243](#243),
[#255](#255),
[#236](#236)).

API changes:
* Renamed `IsAccountsClient` to `IsAccountClient`
([#231](#231)).
* `w.ClusterPolicies.ListAll` now takes `clusterpolicies.List` as an
argument.
* `github.com/databricks/databricks-sdk-go/service/dbsql` package is
renamed to `github.com/databricks/databricks-sdk-go/service/sql`.
 * `w.DataSources.ListDataSources` is renamed to `w.DataSources.List`.
 * `w.Queries.CreateQuery` is renamed to `w.Queries.CreateQuery`.
* `w.Queries.DeleteQueryByQueryId` is renamed to
`w.Queries.DeleteByQueryId`.
 * `w.Queries.GetQueryByQueryId` is renamed to `w.Queries.GetByQueryId`.
 * `w.Queries.UpdateQuery` is renamed to `w.Queries.Update`.
* `w.Alerts.DeleteAlertByAlertId` is renamed to
`w.Alerts.DeleteByAlertId`.
 * `w.Alerts.UpdateAlert` is renamed to `w.Alerts.Update`.
 * `w.Alerts.GetAlertByAlertId` is renamed to `w.Alerts.GetByAlertId`.
 * `w.Alerts.ListAlerts` is renamed to `w.Alerts.List`.
 * `w.Dashboards.CreateDashboard` is renamed to `w.Dashboards.Create`.
* `w.Dashboards.DeleteDashboardByDashboardId` is renamed to
`w.Dashboards.GetByDashboardId`.
* `w.Dashboards.ListDashboardsAll` is renamed to `w.Dashboards.ListAll`.
* `w.Dashboards.DeleteDashboardByDashboardId` is renamed to
`w.Dashboards.DeleteByDashboardId`.
 * `w.Dashboards.RestoreDashboard` is renamed to `w.Dashboards.Restore`.
* `deployment.CreateCustomerManagedKeyRequest` now takes
`deployment.KeyUseCase` enum.
* `w.GlobalInitScripts.CreateScript` is renamed to
`w.GlobalInitScripts.Create`.
* `w.GlobalInitScripts.DeleteScriptByScriptId` is renamed to
`w.GlobalInitScripts.DeleteByScriptId`.
* `w.GlobalInitScripts.UpdateScript` is renamed to
`w.GlobalInitScripts.Update`.
* `w.GlobalInitScripts.GetScriptByScriptId` is renamed to
`w.GlobalInitScripts.GetByScriptId`.
* `w.GlobalInitScripts.ListScriptsAll` is renamed to
`w.GlobalInitScripts.ListAll`.
 * `jobs.ResetJob.NewSettings` is now required field.
 * `w.Pipelines.CreatePipeline` is renamed to `w.Pipelines.Create`.
* `w.Pipelines.DeletePipelineByPipelineId` is renamed to
`w.Pipelines.DeleteByPipelineId`.
 * `w.Pipelines.UpdatePipeline` is renamed to `w.Pipelines.Update`.
* `w.Pipelines.GetPipelineByPipelineId` is renamed to
`w.Pipelines.GetByPipelineId`.
 * `w.StorageCredentials.Update` now also returns an entity.
 * `w.ExternalLocations.Update` now also returns an entity.
 * `w.Metastores.Update` now also returns an entity.
* `unitycatalog.CreateMetastoreAssignment.WorkspaceId` type changed from
`int` to `int64`.
* `unitycatalog.UnassignRequest.WorkspaceId` type changed from `int` to
`int64`.
 * `w.Catalogs.Update` now also returns an entity.
 * `w.Schemas.Update` now also returns an entity.
 * `w.Providers.Update` now also returns an entity.
 * `w.Shares.Update` now also returns an entity.
* `WarehousesAPI` service moved to
`github.com/databricks/databricks-sdk-go/service/sql` package.
* `w.Warehouses.CreateWarehouseAndWait` renamed to
`w.Warehouses.CreateAndWait`.
* `w.Warehouses.DeleteWarehouseByIdAndWait` renamed to
`w.Warehouses.DeleteByIdAndWait`.
 * `w.Warehouses.EditWarehouse` renamed to `w.Warehouses.Edit`.
 * `w.Warehouses.GetWarehouseById` renamed to `w.Warehouses.GetById`.
 * `w.Warehouses.ListWarehousesAll` renamed to `w.Warehouses.ListAll`.
* Removed `w.Dbfs.Overwrite` in favor of `w.Dbfs.Open("....",
dbfs.FileModeOverwrite|dbfs.FileModeWrite)`.
 * Added third required argument to `w.Dbfs.Open`.

Code generation:

* Added concept of `main` service for the package
([#239](#239)).
* Added entity primitives check
([#242](#242)).
* Added helpers for CRUD generation
([#246](#246)).
* Added more entity-generation utils
([#257](#257)).
* Dynamically generate `.gitattributes`
([#244](#244)).
* Fixed required order fields
([#245](#245)).
* Parse summary from descriptions
([#228](#228)).
* Print error on formatter failure
([#235](#235)).
* Update usage string in generator
([#260](#260)).
* Fixed order of host completion
([#233](#233)).

Dependency updates:

* Bump google.golang.org/api from 0.103.0 to 0.105.0
([#232](#232),
[#252](#252)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ability to tell loop iteration in retries.Info

3 participants