Skip to content

Apply attribute transformer when reading in attributes from the environment#293

Merged
mgyucht merged 3 commits intomainfrom
issue-292
Aug 17, 2023
Merged

Apply attribute transformer when reading in attributes from the environment#293
mgyucht merged 3 commits intomainfrom
issue-292

Conversation

@mgyucht
Copy link
Copy Markdown
Contributor

@mgyucht mgyucht commented Aug 17, 2023

Changes

Attributes read from the environment are not currently parsed before being stored in the _inner field of Config. As a result, the type of the attribute will be different depending on whether the attribute is read from the environment or passed to the constructor. This PR changes all argument parsing to use __setattr__ which converts the argument to the expected type.

Closes #292.

Tests

Unit test to ensure that string env vars are parsed to appropriate type for the affected field.

  • make test run locally
  • make fmt applied
  • relevant integration tests applied

@mgyucht mgyucht requested a review from pietern August 17, 2023 13:18
@mgyucht mgyucht added this pull request to the merge queue Aug 17, 2023
Merged via the queue into main with commit 2f1809d Aug 17, 2023
@mgyucht mgyucht deleted the issue-292 branch August 17, 2023 14:45
mgyucht added a commit that referenced this pull request Aug 17, 2023
* Added collection of Databricks Runtime versions used together with Python SDK ([#287](#287)).
* Applied attribute transformer when reading in attributes from the environment ([#293](#293)).
* Made notebook-native auth work with more configurations of the Databricks Runtime ([#285](#285)).
* Added retry in `w.clusters.ensure_cluster_is_running(id)` when cluster is simultaneously started by two different processes. ([#283](#283)).
* Set necessary headers when authenticating via Azure CLI ([#290](#290)).
* Updated classifier to `Development Status :: 4 - Beta` ([#291](#291)).
* Introduced Artifact Allowlist, Securable Tags, and Subentity Tags services.
* Introduced DeleteRuns and RestoreRuns methods in the Experiments API.
* Introduced the GetSecret method in the Secrets API.
* Renamed Auto Maintenance to Predictive Optimization.

New Services:

 * Added [w.artifact_allowlists](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/artifact_allowlists.html) workspace-level service.
 * Added [w.securable_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/securable_tags.html) workspace-level service.
 * Added [w.subentity_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/subentity_tags.html) workspace-level service.
 * Added `databricks.sdk.service.catalog.ArtifactAllowlistInfo` dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactMatcher` dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactType` dataclass.
 * Added `databricks.sdk.service.catalog.GetArtifactAllowlistRequest` dataclass.
 * Added `databricks.sdk.service.catalog.ListSecurableTagsRequest` dataclass.
 * Added `databricks.sdk.service.catalog.ListSecurableType` dataclass.
 * Added `databricks.sdk.service.catalog.ListSubentityTagsRequest` dataclass.
 * Added `databricks.sdk.service.catalog.MatchType` dataclass.
 * Added `databricks.sdk.service.catalog.SetArtifactAllowlist` dataclass.
 * Added `databricks.sdk.service.catalog.TagChanges` dataclass.
 * Added `databricks.sdk.service.catalog.TagKeyValuePair` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurable` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurableAssignment` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurableAssignmentsList` dataclass.
 * Added `databricks.sdk.service.catalog.TagSubentity` dataclass.
 * Added `databricks.sdk.service.catalog.TagSubentityAssignmentsList` dataclass.
 * Added `databricks.sdk.service.catalog.TagsSubentityAssignment` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateSecurableType` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateTags` dataclass.

New APIs:

 * Added `delete_runs()` method for [w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html) workspace-level service.
 * Added `restore_runs()` method for [w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html) workspace-level service.
 * Added `databricks.sdk.service.ml.DeleteRuns` dataclass.
 * Added `databricks.sdk.service.ml.DeleteRunsResponse` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRuns` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRunsResponse` dataclass.
 * Added `get_secret()` method for [w.secrets](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/secrets.html) workspace-level service.
 * Added `databricks.sdk.service.workspace.GetSecretRequest` dataclass.
 * Added `databricks.sdk.service.workspace.GetSecretResponse` dataclass.

Service Renames:

 * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Added `effective_predictive_optimization_flag` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Added `enable_predictive_optimization` field for `databricks.sdk.service.catalog.CatalogInfo`.
 * Removed `databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlag` dataclass.
 * Removed `databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlagInheritedFromType` dataclass.
 * Removed `databricks.sdk.service.catalog.EnableAutoMaintenance` dataclass.
 * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Added `effective_predictive_optimization_flag` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Added `enable_predictive_optimization` field for `databricks.sdk.service.catalog.SchemaInfo`.
 * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.TableInfo`.
 * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.TableInfo`.
 * Added `effective_predictive_optimization_flag` field for `databricks.sdk.service.catalog.TableInfo`.
 * Added `enable_predictive_optimization` field for `databricks.sdk.service.catalog.TableInfo`.
 * Added `databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlag` dataclass.
 * Added `databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlagInheritedFromType` dataclass.
 * Added `databricks.sdk.service.catalog.EnablePredictiveOptimization` dataclass.

OpenAPI SHA: beff621d7b3e1d59244e2e34fc53a496f310e130, Date: 2023-08-17
@mgyucht mgyucht mentioned this pull request Aug 17, 2023
github-merge-queue bot pushed a commit that referenced this pull request Aug 17, 2023
* Added collection of Databricks Runtime versions used together with
Python SDK
([#287](#287)).
* Applied attribute transformer when reading in attributes from the
environment
([#293](#293)).
* Made notebook-native auth work with more configurations of the
Databricks Runtime
([#285](#285)).
* Added retry in `w.clusters.ensure_cluster_is_running(id)` when cluster
is simultaneously started by two different processes.
([#283](#283)).
* Set necessary headers when authenticating via Azure CLI
([#290](#290)).
* Updated classifier to `Development Status :: 4 - Beta`
([#291](#291)).
* Introduced Artifact Allowlist, Securable Tags, and Subentity Tags
services.
* Introduced DeleteRuns and RestoreRuns methods in the Experiments API.
* Introduced the GetSecret method in the Secrets API.
* Renamed Auto Maintenance to Predictive Optimization.

New Services:

* Added
[w.artifact_allowlists](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/artifact_allowlists.html)
workspace-level service.
* Added
[w.securable_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/securable_tags.html)
workspace-level service.
* Added
[w.subentity_tags](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/subentity_tags.html)
workspace-level service.
* Added `databricks.sdk.service.catalog.ArtifactAllowlistInfo`
dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactMatcher` dataclass.
 * Added `databricks.sdk.service.catalog.ArtifactType` dataclass.
* Added `databricks.sdk.service.catalog.GetArtifactAllowlistRequest`
dataclass.
* Added `databricks.sdk.service.catalog.ListSecurableTagsRequest`
dataclass.
 * Added `databricks.sdk.service.catalog.ListSecurableType` dataclass.
* Added `databricks.sdk.service.catalog.ListSubentityTagsRequest`
dataclass.
 * Added `databricks.sdk.service.catalog.MatchType` dataclass.
* Added `databricks.sdk.service.catalog.SetArtifactAllowlist` dataclass.
 * Added `databricks.sdk.service.catalog.TagChanges` dataclass.
 * Added `databricks.sdk.service.catalog.TagKeyValuePair` dataclass.
 * Added `databricks.sdk.service.catalog.TagSecurable` dataclass.
* Added `databricks.sdk.service.catalog.TagSecurableAssignment`
dataclass.
* Added `databricks.sdk.service.catalog.TagSecurableAssignmentsList`
dataclass.
 * Added `databricks.sdk.service.catalog.TagSubentity` dataclass.
* Added `databricks.sdk.service.catalog.TagSubentityAssignmentsList`
dataclass.
* Added `databricks.sdk.service.catalog.TagsSubentityAssignment`
dataclass.
 * Added `databricks.sdk.service.catalog.UpdateSecurableType` dataclass.
 * Added `databricks.sdk.service.catalog.UpdateTags` dataclass.

New APIs:

* Added `delete_runs()` method for
[w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html)
workspace-level service.
* Added `restore_runs()` method for
[w.experiments](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/experiments.html)
workspace-level service.
 * Added `databricks.sdk.service.ml.DeleteRuns` dataclass.
 * Added `databricks.sdk.service.ml.DeleteRunsResponse` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRuns` dataclass.
 * Added `databricks.sdk.service.ml.RestoreRunsResponse` dataclass.
* Added `get_secret()` method for
[w.secrets](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/secrets.html)
workspace-level service.
 * Added `databricks.sdk.service.workspace.GetSecretRequest` dataclass.
 * Added `databricks.sdk.service.workspace.GetSecretResponse` dataclass.

Service Renames:

* Removed `effective_auto_maintenance_flag` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Removed `enable_auto_maintenance` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Added `effective_predictive_optimization_flag` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Added `enable_predictive_optimization` field for
`databricks.sdk.service.catalog.CatalogInfo`.
* Removed `databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlag`
dataclass.
* Removed
`databricks.sdk.service.catalog.EffectiveAutoMaintenanceFlagInheritedFromType`
dataclass.
* Removed `databricks.sdk.service.catalog.EnableAutoMaintenance`
dataclass.
* Removed `effective_auto_maintenance_flag` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Removed `enable_auto_maintenance` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Added `effective_predictive_optimization_flag` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Added `enable_predictive_optimization` field for
`databricks.sdk.service.catalog.SchemaInfo`.
* Removed `effective_auto_maintenance_flag` field for
`databricks.sdk.service.catalog.TableInfo`.
* Removed `enable_auto_maintenance` field for
`databricks.sdk.service.catalog.TableInfo`.
* Added `effective_predictive_optimization_flag` field for
`databricks.sdk.service.catalog.TableInfo`.
* Added `enable_predictive_optimization` field for
`databricks.sdk.service.catalog.TableInfo`.
* Added
`databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlag`
dataclass.
* Added
`databricks.sdk.service.catalog.EffectivePredictiveOptimizationFlagInheritedFromType`
dataclass.
* Added `databricks.sdk.service.catalog.EnablePredictiveOptimization`
dataclass.

OpenAPI SHA: beff621d7b3e1d59244e2e34fc53a496f310e130, Date: 2023-08-17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ISSUE] Function calls are crashing with an error if DATABRICKS_DEBUG_TRUNCATE_BYTES environment variable is set

3 participants