Skip to content

[Internal] Migrate imports for terraform plugin framework + update init test provider factory#3943

Merged
edwardfeng-db merged 3 commits intomainfrom
edwardfeng-db/migrate-imports
Aug 29, 2024
Merged

[Internal] Migrate imports for terraform plugin framework + update init test provider factory#3943
edwardfeng-db merged 3 commits intomainfrom
edwardfeng-db/migrate-imports

Conversation

@edwardfeng-db
Copy link
Copy Markdown
Contributor

@edwardfeng-db edwardfeng-db commented Aug 23, 2024 •
edited
Loading

Changes

  • Migrated imports following the documentation here
image
  • Made run in init_test able to take both plugin framework resources as well as sdkv2 resources
  • Updated providers.GetProviderServer to take in override options used in TestMwsAccAwsChangeToServicePrincipal for per-step provider factory override

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

@edwardfeng-db edwardfeng-db requested review from a team as code owners August 23, 2024 14:52
@edwardfeng-db edwardfeng-db requested review from hectorcast-db and removed request for a team August 23, 2024 14:52
@edwardfeng-db edwardfeng-db changed the title [internal] Migrate imports for terraform plugin framework [Internal] Migrate imports for terraform plugin framework + update init test provider factory Aug 23, 2024
@edwardfeng-db edwardfeng-db marked this pull request as draft August 23, 2024 16:03
@edwardfeng-db edwardfeng-db marked this pull request as ready for review August 27, 2024 14:49
@edwardfeng-db edwardfeng-db force-pushed the edwardfeng-db/migrate-imports branch from df4dc0f to fc3c6a7 Compare August 27, 2024 14:58
@edwardfeng-db
Copy link
Copy Markdown
Contributor Author

edwardfeng-db commented Aug 27, 2024

Integration tests don't have failures related to this infra import change.

tanmay-db
tanmay-db approved these changes Aug 27, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@tanmay-db tanmay-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had one comment, rest LGTM

Comment thread internal/providers/providers.go

// WithSdkV2Provider takes in a *schema.Provider from sdk v2 and returns a ServerOption.
// This is typically used in acceptance test for a test step to have a custom provider override.
func WithSdkV2Provider(sdkV2Provider *schema.Provider) ServerOption {
Copy link
Copy Markdown
Contributor

@tanmay-db tanmay-db Aug 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be only used in internal/acceptance/mws_workspaces_test.go? If yes, then probably best to move it to providers_test_utils.go

Copy link
Copy Markdown
Contributor

@mgyucht mgyucht Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to stay here since it is part of the (internal) public interface for GetProviderServer. Alternatively, we could have two copies of GetProviderServer(), one with only the context argument and a customizable one, but that seems redundant to me.

@edwardfeng-db
Copy link
Copy Markdown
Contributor Author

edwardfeng-db commented Aug 28, 2024

Update - holding off the merge. There's actually some other acceptance test failure I noticed that's related to this. Investigating

mgyucht
mgyucht reviewed Aug 28, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@mgyucht mgyucht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make sure all integration tests pass before merging this.

Comment thread go.sum
Comment thread internal/providers/providers.go
Comment thread internal/providers/providers.go Outdated
Comment thread internal/providers/providers.go Outdated
Comment thread internal/providers/providers.go

// WithSdkV2Provider takes in a *schema.Provider from sdk v2 and returns a ServerOption.
// This is typically used in acceptance test for a test step to have a custom provider override.
func WithSdkV2Provider(sdkV2Provider *schema.Provider) ServerOption {
Copy link
Copy Markdown
Contributor

@mgyucht mgyucht Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to stay here since it is part of the (internal) public interface for GetProviderServer. Alternatively, we could have two copies of GetProviderServer(), one with only the context argument and a customizable one, but that seems redundant to me.

@edwardfeng-db edwardfeng-db force-pushed the edwardfeng-db/migrate-imports branch from 9f51803 to e7e806b Compare August 28, 2024 10:28
@edwardfeng-db edwardfeng-db added this pull request to the merge queue Aug 29, 2024
Merged via the queue into main with commit b2c989a Aug 29, 2024
@edwardfeng-db edwardfeng-db deleted the edwardfeng-db/migrate-imports branch August 29, 2024 10:03
@hectorcast-db hectorcast-db mentioned this pull request Sep 12, 2024
Closed
hectorcast-db added a commit that referenced this pull request Sep 12, 2024
@hectorcast-db
[Release] Release v1.52.0
5206f57 
### New Features and Improvements

 * Added `no_wait` option for clusters to skip waiting to start on cluster creation ([#3953](#3953)).
 * Introduced Plugin Framework ([#3920](#3920)).

### Bug Fixes

 * Add suppress diff for `azure_attributes.spot_bid_max_price` in `databricks_instance_pool` ([#3970](#3970)).
 * Correctly send workload_type fields in `databricks_cluster` to allow users to disable usage in certain contexts ([#3972](#3972)).
 * Fix `TestAccClusterResource_WorkloadType` ([#3989](#3989)).
 * Fix `databricks_sql_table` treatment of properties ([#3925](#3925)).
 * Force send fields for settings resources ([#3978](#3978)).
 * Handle cluster deletion in `databricks_library` read ([#3909](#3909)).
 * Make subscriptions optional for SqlAlertTask ([#3983](#3983)).

### Documentation

 * Add troubleshooting guide for Provider "registry.terraform.io/databricks/databricks" planned an invalid value ([#3961](#3961)).
 * Adopt official naming of Mosaic AI Vector Search ([#3971](#3971)).
 * Document Terraform 1.0 as minimum version ([#3952](#3952)).
 * Mention Salesforce as supported type in `databricks_connection` ([#3949](#3949)).
 * Reimplement Azure Databricks deployment guide to use VNet injection & NPIP ([#3986](#3986)).
 * Resolves [#3127](#3127): Remove deprecated account_id field from mws_credentials resource ([#3974](#3974)).
 * Small Grammar Corrections in Docs ([#4006](#4006)).
 * Update `databricks_vector_search_index` docs to match latest SDK ([#4008](#4008)).
 * Update aws_unity_catalog_assume_role_policy.md ([#3968](#3968)).
 * Update documentation regarding authentication with Azure-managed Service Principal using GITHUB OIDC ([#3932](#3932)).
 * Update metastore_assignment.md to properly reflect possible usage ([#3967](#3967)).
 * Update minimum supported terraform version to 1.1.5 ([#3965](#3965)).
 * Update resources diagram to include newer resources ([#3962](#3962)).
 * Update workspace_binding import command ([#3944](#3944)).
 * fix possible values for `securable_type` in `databricks_workspace_binding` ([#3942](#3942)).

### Internal Changes

 * Add `AddPlanModifer` method for AttributeBuilder ([#4009](#4009)).
 * Add integration tests for volumes and quality monitor plugin framework ([#3975](#3975)).
 * Add support for `computed` tag in TfSDK Structs ([#4005](#4005)).
 * Added `databricks_quality_monitor` resource and `databricks_volumes` data source to plugin framework ([#3958](#3958)).
 * Allow vector search tests to fail ([#3959](#3959)).
 * Make test utils public and move integration test for quality monitor ([#3993](#3993)).
 * Migrate Share resource to Go SDK ([#3916](#3916)).
 * Migrate imports for terraform plugin framework + update init test provider factory ([#3943](#3943)).
 * Move volumes test next to plugin framework data source ([#3995](#3995)).
 * Refactor provider and related packages ([#3940](#3940)).
 * Support import in acceptance test + adding import state for quality monitor ([#3994](#3994)).

### Dependency Updates

 * Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0 ([#3948](#3948)).
 * Update Go SDK to 0.46.0 ([#4007](#4007)).

### Exporter

 * Don't generate instance pools if the pool name is empty ([#3960](#3960)).
@tanmay-db tanmay-db mentioned this pull request Sep 16, 2024
Merged
tanmay-db added a commit that referenced this pull request Sep 16, 2024
@tanmay-db
[Release] Release v1.52.0
c83c514 
### New Features and Improvements

 * Add support for filters in `databricks_clusters` data source ([#4014](#4014)).
 * Added `no_wait` option for clusters to skip waiting to start on cluster creation ([#3953](#3953)).
 * Introduced Plugin Framework ([#3920](#3920)).
 * Library plugin framework migration ([#3979](#3979)).

### Bug Fixes

 * Add suppress diff for `azure_attributes.spot_bid_max_price` in `databricks_instance_pool` ([#3970](#3970)).
 * Correctly send workload_type fields in `databricks_cluster` to allow users to disable usage in certain contexts ([#3972](#3972)).
 * Fix `TestAccClusterResource_WorkloadType` ([#3989](#3989)).
 * Fix `databricks_sql_table` treatment of properties ([#3925](#3925)).
 * Force send fields for settings resources ([#3978](#3978)).
 * Handle cluster deletion in `databricks_library` read ([#3909](#3909)).
 * Make subscriptions optional for SqlAlertTask ([#3983](#3983)).
 * Permanently delete `ERROR` and `TERMINATED` state clusters if their creation fails ([#4021](#4021)).

### Documentation

 * Add troubleshooting guide for Provider "registry.terraform.io/databricks/databricks" planned an invalid value ([#3961](#3961)).
 * Adopt official naming of Mosaic AI Vector Search ([#3971](#3971)).
 * Document Terraform 1.0 as minimum version ([#3952](#3952)).
 * Mention Salesforce as supported type in `databricks_connection` ([#3949](#3949)).
 * Reimplement Azure Databricks deployment guide to use VNet injection & NPIP ([#3986](#3986)).
 * Resolves [#3127](#3127): Remove deprecated account_id field from mws_credentials resource ([#3974](#3974)).
 * Small Grammar Corrections in Docs ([#4006](#4006)).
 * Update `databricks_vector_search_index` docs to match latest SDK ([#4008](#4008)).
 * Update aws_unity_catalog_assume_role_policy.md ([#3968](#3968)).
 * Update documentation regarding authentication with Azure-managed Service Principal using GITHUB OIDC ([#3932](#3932)).
 * Update metastore_assignment.md to properly reflect possible usage ([#3967](#3967)).
 * Update minimum supported terraform version to 1.1.5 ([#3965](#3965)).
 * Update resources diagram to include newer resources ([#3962](#3962)).
 * Update workspace_binding import command ([#3944](#3944)).
 * fix possible values for `securable_type` in `databricks_workspace_binding` ([#3942](#3942)).

### Internal Changes

 * Add `AddPlanModifer` method for AttributeBuilder ([#4009](#4009)).
 * Add integration tests for volumes and quality monitor plugin framework ([#3975](#3975)).
 * Add support for `computed` tag in TfSDK Structs ([#4005](#4005)).
 * Added `databricks_quality_monitor` resource and `databricks_volumes` data source to plugin framework ([#3958](#3958)).
 * Allow vector search tests to fail ([#3959](#3959)).
 * Clean up comments in library resource ([#4015](#4015)).
 * Fix irregularities in plugin framework converter function errors ([#4010](#4010)).
 * Make test utils public and move integration test for quality monitor ([#3993](#3993)).
 * Migrate Share resource to Go SDK ([#3916](#3916)).
 * Migrate `databricks_cluster` data source to plugin framework ([#3988](#3988)).
 * Migrate imports for terraform plugin framework + update init test provider factory ([#3943](#3943)).
 * Move volumes test next to plugin framework data source ([#3995](#3995)).
 * Refactor provider and related packages ([#3940](#3940)).
 * Support import in acceptance test + adding import state for quality monitor ([#3994](#3994)).

### Dependency Updates

 * Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0 ([#3948](#3948)).
 * Update Go SDK to 0.46.0 ([#4007](#4007)).

### Exporter

 * Don't generate instance pools if the pool name is empty ([#3960](#3960)).
github-merge-queue bot pushed a commit that referenced this pull request Sep 17, 2024
@tanmay-db
[Release] Release v1.52.0 (#4022)
e13ce2c 
### New Features and Improvements

* Add support for filters in `databricks_clusters` data source
([#4014](#4014)).
* Added `no_wait` option for clusters to skip waiting to start on
cluster creation
([#3953](#3953)).
* Introduced Plugin Framework
([#3920](#3920)).


### Bug Fixes

* Add suppress diff for `azure_attributes.spot_bid_max_price` in
`databricks_instance_pool`
([#3970](#3970)).
* Correctly send workload_type fields in `databricks_cluster` to allow
users to disable usage in certain contexts
([#3972](#3972)).
* Fix `databricks_sql_table` treatment of properties
([#3925](#3925)).
* Force send fields for settings resources
([#3978](#3978)).
* Handle cluster deletion in `databricks_library` read
([#3909](#3909)).
* Make subscriptions optional for SqlAlertTask
([#3983](#3983)).
* Permanently delete `ERROR` and `TERMINATED` state clusters if their
creation fails
([#4021](#4021)).


### Documentation

* Add troubleshooting guide for Provider
"registry.terraform.io/databricks/databricks" planned an invalid value
([#3961](#3961)).
* Adopt official naming of Mosaic AI Vector Search
([#3971](#3971)).
* Document Terraform 1.0 as minimum version
([#3952](#3952)).
* Mention Salesforce as supported type in `databricks_connection`
([#3949](#3949)).
* Reimplement Azure Databricks deployment guide to use VNet injection &
NPIP
([#3986](#3986)).
* Resolves
[#3127](#3127):
Remove deprecated account_id field from mws_credentials resource
([#3974](#3974)).
* Small Grammar Corrections in Docs
([#4006](#4006)).
* Update `databricks_vector_search_index` docs to match latest SDK
([#4008](#4008)).
* Update aws_unity_catalog_assume_role_policy.md
([#3968](#3968)).
* Update documentation regarding authentication with Azure-managed
Service Principal using GITHUB OIDC
([#3932](#3932)).
* Update metastore_assignment.md to properly reflect possible usage
([#3967](#3967)).
* Update minimum supported terraform version to 1.1.5
([#3965](#3965)).
* Update resources diagram to include newer resources
([#3962](#3962)).
* Update workspace_binding import command
([#3944](#3944)).
* fix possible values for `securable_type` in
`databricks_workspace_binding`
([#3942](#3942)).


### Internal Changes

* Add `AddPlanModifer` method for AttributeBuilder
([#4009](#4009)).
* Add integration tests for volumes and quality monitor plugin framework
([#3975](#3975)).
* Add support for `computed` tag in TfSDK Structs
([#4005](#4005)).
* Added `databricks_quality_monitor` resource and `databricks_volumes`
data source to plugin framework
([#3958](#3958)).
* Allow vector search tests to fail
([#3959](#3959)).
* Clean up comments in library resource
([#4015](#4015)).
* Fix irregularities in plugin framework converter function errors
([#4010](#4010)).
* Make test utils public and move integration test for quality monitor
([#3993](#3993)).
* Migrate Share resource to Go SDK
([#3916](#3916)).
* Migrate `databricks_cluster` data source to plugin framework
([#3988](#3988)).
* Migrate imports for terraform plugin framework + update init test
provider factory
([#3943](#3943)).
* Move volumes test next to plugin framework data source
([#3995](#3995)).
* Refactor provider and related packages
([#3940](#3940)).
* Support import in acceptance test + adding import state for quality
monitor
([#3994](#3994)).
* Library plugin framework migration
([#3979](#3979)).
* Fix `TestAccClusterResource_WorkloadType`
([#3989](#3989)).


### Dependency Updates

* Bump github.com/hashicorp/hcl/v2 from 2.21.0 to 2.22.0
([#3948](#3948)).
* Update Go SDK to 0.46.0
([#4007](#4007)).


### Exporter

* Don't generate instance pools if the pool name is empty
([#3960](#3960)).
* Expand list of non-interactive clusters
([#4023](#4023)).
* Ignore databricks_artifact_allowlist with zero artifact_matcher blocks
([#4019](#4019)).


## [Release] Release v1.51.0

### Breaking Changes

With this release, only protocol version 6 will be supported which is
compatible with terraform CLI version 1.1.5 and later. If you are using
an older version of the terraform CLI, please upgrade it to use this and
further releases of Databricks terraform provider.

### New Features and Improvements

* Automatically create `parent_path` folder when creating
`databricks_dashboard resource` if it doesn't exist
([#3778](#3778)).


### Bug Fixes

* Fixed logging for underlying Go SDK
([#3917](#3917)).
* Remove not necessary field in `databricks_job` schema
([#3907](#3907)).


### Internal Changes

* Add AttributeBuilder for Plugin Framework schema
([#3922](#3922)).
* Add CustomizableSchema for Plugin Framework
([#3927](#3927)).
* Add StructToSchema for Plugin Framework
([#3928](#3928)).
* Add codegen template and generated files for tfsdk structs
([#3911](#3911)).
* Add converter functions and tests for plugin framework
([#3914](#3914)).
* Added support to use protocol version 6 provider server for SDK plugin
([#3862](#3862)).
* Bump Go SDK to v0.45.0
([#3933](#3933)).
* Change name with the aliases in codegen template
([#3936](#3936)).
* Update jd version from latest to 1.8.1
([#3915](#3915)).
* Upgrade `staticcheck` to v0.5.1 to get Go 1.23 support
([#3931](#3931)).
* OPENAPI_SHA check
([#3935](#3935)).
* Use generic error for missing clusters
([#3938](#3938))


### Exporter

* Better support for notebooks with /Workspace path
([#3901](#3901)).
* Improve exporting of DLT and test coverage
([#3898](#3898)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tanmay-db tanmay-db tanmay-db approved these changes

@hectorcast-db hectorcast-db Awaiting requested review from hectorcast-db hectorcast-db is a code owner automatically assigned from databricks/eng-plat-auto-exp-reviewers

+1 more reviewer

@mgyucht mgyucht mgyucht left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@edwardfeng-db @mgyucht @tanmay-db