[DECO-1115] Add local implementation for `dbutils.widgets` by kartikgupta-db · Pull Request #93 · databricks/databricks-sdk-py

kartikgupta-db · 2023-05-05T11:24:07Z

Changes

Added a new install group (pip install 'databricks-sdk[notebook]'). This allows us to safely pin ipywidgets for local installs. DBR can safely continue using pip install databricks-sdk or directly using the default build from master without conflicting dependencies.
OSS implementation of widgets is imported only on first use (possible only through OSS implementation of dbutils - RemoteDbutils).
Add a wrapper for ipywidgets to enable interactive widgets when in interactive IPython notebooks.

Screen.Recording.2023-05-05.at.16.37.29.mov

Add default widgets implementation that returns a default value, when not in an interactive environment.

Screen.Recording.2023-05-05.at.16.39.40.mov

Tests

make test run locally
make fmt applied
relevant integration tests applied

…upyter-wdgets

kartikgupta-db · 2023-05-05T11:24:45Z

TODO: tests

tests/test_dbutils.py

databricks/sdk/_widgets/__init__.py

databricks/sdk/_widgets/IPyWidgetUtils.py

databricks/sdk/_widgets/WidgetUtils.py

nfx · 2023-05-05T13:51:02Z

databricks/sdk/_widgets/__init__.py

+import os
+
+try:
+    from IPython.core.getipython import get_ipython


what about working in DBR?

On dbr, dbutils entirely defaults to the builtin implementation. So this widget implementation is not used anyway.

Having said that, this still works for dbr notebooks because they are based on iPython.

In DBR, we will need to find some way override dbutils.widgets by the DBR-specific implementation. I think it's better to do that in universe during the REPL startup time

@xiaochen-db we do not use the this version (the oss version) of dbutils within dbr at all (#96). So we shouldn't need to override widgets specifically. Same for ipywidget versioning. The oss implementation is activated only when outside dbr (when a default global implementation of dbutils is not available).

databricks/sdk/dbutils.py

xiaochen-db · 2023-05-09T04:51:12Z

databricks/sdk/_widgets/__init__.py

+import os
+
+try:
+    from IPython.core.getipython import get_ipython


In DBR, we will need to find some way override dbutils.widgets by the DBR-specific implementation. I think it's better to do that in universe during the REPL startup time

xiaochen-db · 2023-05-09T04:57:19Z

databricks/sdk/_widgets/IPyWidgetUtils.py

+import typing
+
+from IPython.core.display_functions import display
+from ipywidgets.widgets import (ValueWidget, Widget, widget_box,


Naive question: did we specific the version of ipywidgets? When DBR uses a different version for ipywidgets, who wins?

Users can access dbutils from 2 points

from databricks.sdk.runtime import *: This always falls back to the dbr version of dbutils. The oss implementations do not come into effect.

workspace_client.dbutils: This always uses the open source version of dbutils. We are working on a fix so that this property also falls back to the dbr version of dbutils when possible.
This means that any imports for ipywidgets and other deps do not take effect when on dbr.

Further, I added a new install group (databricks-sdk[notebook]), which has certain pinned versions for widgets and ipython. Users are prompted to install it if we detect that users are in an interactive notebook environment, but by default this will not be installed on the cluster. These checks are also bypassed in the dbr in favour of dbr native dbutils.

databricks/sdk/_widgets/IPyWidgetUtils.py

…upyter-wdgets

codecov-commenter · 2023-06-21T22:29:40Z

Codecov Report

Patch coverage: 2.56% and project coverage change: -0.33 ⚠️

Comparison is base (befbb42) 53.40% compared to head (9a03c3e) 53.08%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #93      +/-   ##
==========================================
- Coverage   53.40%   53.08%   -0.33%     
==========================================
  Files          29       32       +3     
  Lines       18441    18558     +117     
==========================================
+ Hits         9848     9851       +3     
- Misses       8593     8707     +114

Impacted Files	Coverage Δ
databricks/sdk/_widgets/__init__.py	`0.00% <0.00%> (ø)`
databricks/sdk/_widgets/default_widgets_utils.py	`0.00% <0.00%> (ø)`
databricks/sdk/_widgets/ipywidgets_utils.py	`0.00% <0.00%> (ø)`
databricks/sdk/dbutils.py	`78.35% <42.85%> (-1.33%)`	⬇️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

mgyucht

Overall LGTM, just a couple nits/questions but feel free to merge after answering/addressing.

setup.py

databricks/sdk/_widgets/__init__.py

mgyucht · 2023-06-28T13:16:44Z

databricks/sdk/_widgets/__init__.py

+    # from REPL or file, this check will fail, since the mro only contains TerminalInteractiveShell.
+    if len(list(filter(lambda i: i.__name__ == 'ZMQInteractiveShell', get_ipython().__class__.__mro__))) == 0:
+        logging.debug("Not in an interactive notebook. Skipping ipywidgets implementation for dbutils.")
+        raise EnvironmentError("Not in an interactive notebook.")


Not a blocker for this change, but have we thought about how widgets would work for users running programs using the Databricks SDK locally? Current fallback implementation uses the default value only, but perhaps that implementation could look for an environment variable named DB_WIDGET_<NAME> to allow automated use-cases to set widget values.

Good point. Let me think about this offline and we can make this a separate PR. This PR is already getting complicated.

mgyucht · 2023-06-28T13:18:17Z

databricks/sdk/_widgets/__init__.py

+    except ImportError as e:
+        # Since we are certain that we are in an interactive notebook, we can make assumptions about
+        # formatting and make the warning nicer for the user.
+        warnings.warn(
+            "\nTo use databricks widgets interactively in your notebook, please install databricks sdk using:\n"
+            "\tpip install 'databricks-sdk[notebook]'\n"
+            "Falling back to default_value_only implementation for databricks widgets.")
+        logging.debug(f"{e.msg}. Skipping ipywidgets implementation for dbutils.")
+        raise e


When would we actually hit this case? Are there Jupyter notebooks that don't have the ipython dependencies installed?

We can hit this case, if any dependency is too old, or missing some features we depend on (eg Combobox from ipywidgets).

…upyter-wdgets

xiaochen-db · 2023-06-29T23:48:51Z

databricks/sdk/_widgets/__init__.py

+        widget_impl = IPyWidgetUtil
+        logging.debug("Using ipywidgets implementation for dbutils.")
+
+    except ImportError as e:


How can user end up into this? Not installing ipywidgets?

Yes. Or if they are on the wrong version (which doesn't have some packages we depend on).

* Add Issue Templates ([#208](#208)). * Fix error message, ExportFormat -> ImportFormat ([#220](#220)). * Fixed notebook native auth for jobs ([#209](#209)). * Make workspace client also return runtime dbutils when in dbr ([#210](#210)). * Regenerate Python SDK using recent OpenAPI Specification ([#229](#229)). * Replace `datatime.timedelta()` with `datetime.timedelta()` in codebase ([#207](#207)). * Support dod in python sdk ([#212](#212)). * Use .ConstantName defining target enum states for waiters ([#230](#230)). * [DECO-1115] Add local implementation for `dbutils.widgets` ([#93](#93)). API Changes: * Added `update()` method for [w.tables](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/tables.html) workspace-level service. * Added [w.lineage_tracking](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/lineage_tracking.html) workspace-level service. * Removed `comment` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `mask` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `nullable` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `partition_index` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `position` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `type_interval_type` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `type_json` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `type_name` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `type_precision` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `type_scale` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `type_text` field for `databricks.sdk.service.catalog.ColumnInfo`. * Added `catalog_name` field for `databricks.sdk.service.catalog.ColumnInfo`. * Added `has_permission` field for `databricks.sdk.service.catalog.ColumnInfo`. * Added `path` field for `databricks.sdk.service.catalog.ColumnInfo`. * Added `schema_name` field for `databricks.sdk.service.catalog.ColumnInfo`. * Added `table_name` field for `databricks.sdk.service.catalog.ColumnInfo`. * Added `table_type` field for `databricks.sdk.service.catalog.ColumnInfo`. * Removed `databricks.sdk.service.catalog.ColumnMask` dataclass. * Removed `databricks.sdk.service.catalog.DataSourceFormat` dataclass. * Removed `databricks.sdk.service.catalog.DeltaRuntimePropertiesKvPairs` dataclass. * Removed `databricks.sdk.service.catalog.TableConstraintList` dataclass. * Removed `columns` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `comment` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `created_at` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `created_by` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `data_access_configuration_id` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `data_source_format` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `deleted_at` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `delta_runtime_properties_kvpairs` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `effective_auto_maintenance_flag` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `enable_auto_maintenance` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `full_name` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `metastore_id` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `owner` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `properties` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `row_filter` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `sql_path` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `storage_credential_name` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `storage_location` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `table_constraints` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `table_id` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `updated_at` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `updated_by` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `view_definition` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `view_dependencies` field for `databricks.sdk.service.catalog.TableInfo`. * Added `has_permission` field for `databricks.sdk.service.catalog.TableInfo`. * Removed `databricks.sdk.service.catalog.TableRowFilter` dataclass. * Added `databricks.sdk.service.catalog.ColumnLineageRequest` dataclass. * Added `databricks.sdk.service.catalog.DashboardInfo` dataclass. * Added `databricks.sdk.service.catalog.FileInfo` dataclass. * Added `databricks.sdk.service.catalog.GetColumnLineagesResponse` dataclass. * Added `databricks.sdk.service.catalog.GetTableEntityLineageResponse` dataclass. * Added `databricks.sdk.service.catalog.JobInfo` dataclass. * Added `databricks.sdk.service.catalog.LineageInfo` dataclass. * Added `databricks.sdk.service.catalog.NotebookInfo` dataclass. * Added `databricks.sdk.service.catalog.PipelineInfo` dataclass. * Added `databricks.sdk.service.catalog.QueryInfo` dataclass. * Added `databricks.sdk.service.catalog.TableLineageRequest` dataclass. * Added `databricks.sdk.service.catalog.UpdateTableRequest` dataclass. * Added `schema` field for `databricks.sdk.service.iam.PartialUpdate`. * Added `databricks.sdk.service.iam.PatchSchema` dataclass. * Added `trigger_info` field for `databricks.sdk.service.jobs.BaseRun`. * Added `health` field for `databricks.sdk.service.jobs.CreateJob`. * Added `job_source` field for `databricks.sdk.service.jobs.GitSource`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.JobEmailNotifications`. * Added `health` field for `databricks.sdk.service.jobs.JobSettings`. * Added `trigger_info` field for `databricks.sdk.service.jobs.Run`. * Added `run_job_output` field for `databricks.sdk.service.jobs.RunOutput`. * Added `run_job_task` field for `databricks.sdk.service.jobs.RunTask`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `health` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `notification_settings` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.Task`. * Added `run_job_task` field for `databricks.sdk.service.jobs.Task`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.TaskEmailNotifications`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.WebhookNotifications`. * Added `databricks.sdk.service.jobs.JobSource` dataclass. * Added `databricks.sdk.service.jobs.JobSourceDirtyState` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthMetric` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthOperator` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRule` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRules` dataclass. * Added `databricks.sdk.service.jobs.RunJobOutput` dataclass. * Added `databricks.sdk.service.jobs.RunJobTask` dataclass. * Added `databricks.sdk.service.jobs.TriggerInfo` dataclass. * Added `databricks.sdk.service.jobs.WebhookNotificationsOnDurationWarningThresholdExceededItem` dataclass. * Removed `whl` field for `databricks.sdk.service.pipelines.PipelineLibrary`. * Changed `delete_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `read_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `etag` field for `databricks.sdk.service.settings.DeletePersonalComputeSettingRequest` to be required. * Changed `etag` field for `databricks.sdk.service.settings.ReadPersonalComputeSettingRequest` to be required. * Added [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service. * Added `databricks.sdk.service.sharing.CentralCleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomAssetInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalog` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalogUpdate` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCollaboratorInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomNotebookInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomTableInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnMask` dataclass. * Added `databricks.sdk.service.sharing.ColumnTypeName` dataclass. * Added `databricks.sdk.service.sharing.CreateCleanRoom` dataclass. * Added `databricks.sdk.service.sharing.DeleteCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.GetCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.ListCleanRoomsResponse` dataclass. * Added `databricks.sdk.service.sharing.UpdateCleanRoom` dataclass. * Changed `query` field for `databricks.sdk.service.sql.Alert` to `databricks.sdk.service.sql.AlertQuery` dataclass. * Changed `value` field for `databricks.sdk.service.sql.AlertOptions` to `any` dataclass. * Removed `is_db_admin` field for `databricks.sdk.service.sql.User`. * Removed `profile_image_url` field for `databricks.sdk.service.sql.User`. * Added `databricks.sdk.service.sql.AlertQuery` dataclass. OpenAPI SHA: 1f35e7f31d95b7dd0b08df53a22ec9dac18b1c84, Date: 2023-07-15

* Add Issue Templates ([#208](#208)). * Added toolchain configuration to `.codegen.json` ([#236](#236)). * Fix enum deserialization ([#234](#234)). * Fix enum deserialization, take 2 ([#235](#235)). * Fix error message, ExportFormat -> ImportFormat ([#220](#220)). * Fixed notebook native auth for jobs ([#209](#209)). * Make OpenAPI spec location configurable ([#237](#237)). * Make workspace client also return runtime dbutils when in dbr ([#210](#210)). * Regenerate Python SDK using recent OpenAPI Specification ([#229](#229)). * Replace `datatime.timedelta()` with `datetime.timedelta()` in codebase ([#207](#207)). * Support dod in python sdk ([#212](#212)). * Use .ConstantName defining target enum states for waiters ([#230](#230)). * [DECO-1115] Add local implementation for `dbutils.widgets` ([#93](#93)). API Changes: * Added `update()` method for [w.tables](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/tables.html) workspace-level service. * Added `databricks.sdk.service.catalog.UpdateTableRequest` dataclass. * Added `schema` field for `databricks.sdk.service.iam.PartialUpdate`. * Added `databricks.sdk.service.iam.PatchSchema` dataclass. * Added `trigger_info` field for `databricks.sdk.service.jobs.BaseRun`. * Added `health` field for `databricks.sdk.service.jobs.CreateJob`. * Added `job_source` field for `databricks.sdk.service.jobs.GitSource`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.JobEmailNotifications`. * Added `health` field for `databricks.sdk.service.jobs.JobSettings`. * Added `trigger_info` field for `databricks.sdk.service.jobs.Run`. * Added `run_job_output` field for `databricks.sdk.service.jobs.RunOutput`. * Added `run_job_task` field for `databricks.sdk.service.jobs.RunTask`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `health` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `notification_settings` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.Task`. * Added `run_job_task` field for `databricks.sdk.service.jobs.Task`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.TaskEmailNotifications`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.WebhookNotifications`. * Added `databricks.sdk.service.jobs.JobSource` dataclass. * Added `databricks.sdk.service.jobs.JobSourceDirtyState` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthMetric` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthOperator` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRule` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRules` dataclass. * Added `databricks.sdk.service.jobs.RunJobOutput` dataclass. * Added `databricks.sdk.service.jobs.RunJobTask` dataclass. * Added `databricks.sdk.service.jobs.TriggerInfo` dataclass. * Added `databricks.sdk.service.jobs.WebhookNotificationsOnDurationWarningThresholdExceededItem` dataclass. * Removed `whl` field for `databricks.sdk.service.pipelines.PipelineLibrary`. * Changed `delete_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `read_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `etag` field for `databricks.sdk.service.settings.DeletePersonalComputeSettingRequest` to be required. * Changed `etag` field for `databricks.sdk.service.settings.ReadPersonalComputeSettingRequest` to be required. * Added [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service. * Added `databricks.sdk.service.sharing.CentralCleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomAssetInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalog` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalogUpdate` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCollaboratorInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomNotebookInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomTableInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnMask` dataclass. * Added `databricks.sdk.service.sharing.ColumnTypeName` dataclass. * Added `databricks.sdk.service.sharing.CreateCleanRoom` dataclass. * Added `databricks.sdk.service.sharing.DeleteCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.GetCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.ListCleanRoomsResponse` dataclass. * Added `databricks.sdk.service.sharing.UpdateCleanRoom` dataclass. * Changed `query` field for `databricks.sdk.service.sql.Alert` to `databricks.sdk.service.sql.AlertQuery` dataclass. * Changed `value` field for `databricks.sdk.service.sql.AlertOptions` to `any` dataclass. * Removed `is_db_admin` field for `databricks.sdk.service.sql.User`. * Removed `profile_image_url` field for `databricks.sdk.service.sql.User`. * Added `databricks.sdk.service.sql.AlertQuery` dataclass. OpenAPI SHA: e20d2b10a181b1e865716de25f42e86d7e3f0270, Date: 2023-07-17

* Add Issue Templates ([#208](#208)). * Fixed notebook native auth for jobs ([#209](#209)). * Replace `datatime.timedelta()` with `datetime.timedelta()` in codebase ([#207](#207)). * Support dod in python sdk ([#212](#212)). * [DECO-1115] Add local implementation for `dbutils.widgets` ([#93](#93)). * Fix error message, ExportFormat -> ImportFormat ([#220](#220)). * Regenerate Python SDK using recent OpenAPI Specification ([#229](#229)). * Make workspace client also return runtime dbutils when in dbr ([#210](#210)). * Use .ConstantName defining target enum states for waiters ([#230](#230)). * Fix enum deserialization ([#234](#234)). * Fix enum deserialization, take 2 ([#235](#235)). * Added toolchain configuration to `.codegen.json` ([#236](#236)). * Make OpenAPI spec location configurable ([#237](#237)). API Changes: * Added `update()` method for [w.tables](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/tables.html) workspace-level service. * Added `databricks.sdk.service.catalog.UpdateTableRequest` dataclass. * Added `schema` field for `databricks.sdk.service.iam.PartialUpdate`. * Added `databricks.sdk.service.iam.PatchSchema` dataclass. * Added `trigger_info` field for `databricks.sdk.service.jobs.BaseRun`. * Added `health` field for `databricks.sdk.service.jobs.CreateJob`. * Added `job_source` field for `databricks.sdk.service.jobs.GitSource`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.JobEmailNotifications`. * Added `health` field for `databricks.sdk.service.jobs.JobSettings`. * Added `trigger_info` field for `databricks.sdk.service.jobs.Run`. * Added `run_job_output` field for `databricks.sdk.service.jobs.RunOutput`. * Added `run_job_task` field for `databricks.sdk.service.jobs.RunTask`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `health` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `notification_settings` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.Task`. * Added `run_job_task` field for `databricks.sdk.service.jobs.Task`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.TaskEmailNotifications`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.WebhookNotifications`. * Added `databricks.sdk.service.jobs.JobSource` dataclass. * Added `databricks.sdk.service.jobs.JobSourceDirtyState` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthMetric` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthOperator` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRule` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRules` dataclass. * Added `databricks.sdk.service.jobs.RunJobOutput` dataclass. * Added `databricks.sdk.service.jobs.RunJobTask` dataclass. * Added `databricks.sdk.service.jobs.TriggerInfo` dataclass. * Added `databricks.sdk.service.jobs.WebhookNotificationsOnDurationWarningThresholdExceededItem` dataclass. * Removed `whl` field for `databricks.sdk.service.pipelines.PipelineLibrary`. * Changed `delete_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `read_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `etag` field for `databricks.sdk.service.settings.DeletePersonalComputeSettingRequest` to be required. * Changed `etag` field for `databricks.sdk.service.settings.ReadPersonalComputeSettingRequest` to be required. * Added [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service. * Added `databricks.sdk.service.sharing.CentralCleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomAssetInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalog` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalogUpdate` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCollaboratorInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomNotebookInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomTableInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnMask` dataclass. * Added `databricks.sdk.service.sharing.ColumnTypeName` dataclass. * Added `databricks.sdk.service.sharing.CreateCleanRoom` dataclass. * Added `databricks.sdk.service.sharing.DeleteCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.GetCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.ListCleanRoomsResponse` dataclass. * Added `databricks.sdk.service.sharing.UpdateCleanRoom` dataclass. * Changed `query` field for `databricks.sdk.service.sql.Alert` to `databricks.sdk.service.sql.AlertQuery` dataclass. * Changed `value` field for `databricks.sdk.service.sql.AlertOptions` to `any` dataclass. * Removed `is_db_admin` field for `databricks.sdk.service.sql.User`. * Removed `profile_image_url` field for `databricks.sdk.service.sql.User`. * Added `databricks.sdk.service.sql.AlertQuery` dataclass. OpenAPI SHA: e20d2b10a181b1e865716de25f42e86d7e3f0270, Date: 2023-07-17

…s#93) ## Changes * Added a new install group (`pip install 'databricks-sdk[notebook]'`). This allows us to safely pin ipywidgets for local installs. DBR can safely continue using `pip install databricks-sdk` or directly using the default build from master without conflicting dependencies. * OSS implementation of widgets is imported only on first use (possible only through OSS implementation of dbutils - `RemoteDbutils`). * Add a wrapper for ipywidgets to enable interactive widgets when in interactive **IPython** notebooks. https://user-images.githubusercontent.com/88345179/236443693-1c804107-ba21-4296-ba40-2b1e8e062d16.mov * Add default widgets implementation that returns a default value, when not in an interactive environment. https://user-images.githubusercontent.com/88345179/236443729-51185404-4d28-49c6-ade0-a665e154e092.mov ## Tests  - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied

author Kartik Gupta <[email protected]> 1688609182 +0200 committer Spece <[email protected]> 1689629188 -0400 gpgsig -----BEGIN PGP SIGNATURE----- iQGzBAABCAAdFiEEmFkdmlNpeHaohb9psG3XEklX7TkFAmS1sgQACgkQsG3XEklX 7Tl6Cgv9GDR+CQVJBr7SrITDS41HgQLIN/s/o3o/gX7htrYC9LTW95hKSUXmirMO 6+ZfI/rJpgB4nwe7fW47mn+nkXS+RrTO+LVZ3mzhiOunNGyGNix8BpjusjSmIxmB kCfgNLjQ9kLVLN1MRrYD4G4eg5zIkjgdNGMpZ1zxOcxue7Vd+qd5nGusUAiBEZIN ok5bw/VrfEBhYFx8S+XS3+OQpXAECc/tO4jmPghq5LIaP3SnlqS3/8MrYZOAP4kT zvmT+l/ySfAtl49I3tV5gDzB1myu717l2rzyE9JovDg4JIpdB+F3ZkYupgKozw8+ RhdM+4NH+0w+A4NhgGKA+fn80cK2ZW9fNLjSqv5JQa2ppb/LJNs+ZiCKQVmNA/jP dYNI++d3sc+lrNlGHjZPQ8NQBJYxMuOC9OKTLC4vEJQ8Cc2hXUKlAYCqIvTFDoKP a21ynsDe8L+CDAKT6u3jSiUbNUlhl8cck4pFsRbVZYiikuBidtlHHLoXmHjeSayZ N8eiCeeh =15Z+ -----END PGP SIGNATURE----- parent befbb42 author Kartik Gupta <[email protected]> 1688609182 +0200 committer Spece <[email protected]> 1689629116 -0400 gpgsig -----BEGIN PGP SIGNATURE----- iQGzBAABCAAdFiEEmFkdmlNpeHaohb9psG3XEklX7TkFAmS1sbwACgkQsG3XEklX 7TnEJgv+PhwF46QO+N5yEQNdzX8sCi/7pkZePiELGSzjkqEL75wNxrYX22PKaW6V 2ThXk7wczONZYozzadZzB72uZ+jqm5xAtr/QaOZhz8h/xJ79IsuGT+rtA198mCjv k+G/2iZzAb5Jcs08X58YrZJCYPQDPTXmElyRUskMhiO2wjmVgcL80JpHk4UKLBfU 2m3ZJY/ZSBFfBdrHCqHUVfyq8KgC7dcxEApgX4ZNb0eE0wC9PbtrfIzgkrTGuMPE lD0Vp7QIAJozWeO5SJe7HHxSQl3qTSiADjZC2wMQQ4a87eKC6g0hDKt7rTGo75s0 p9UO+MUJHh3/QcHkSIzRYOzkPjLOGIhpnMWAufJNOxRVJdiT0xuDFo2bOZ6JMgRV u2BrGa1ujC+sAgmicuAtBZeC5lf8mKpBgW2/VWFZN7UOMeSxZewgSd9G4xLxGOvt C2htT9yH4zsjHZLAlUqQp8LN4QjCo3lSB/0CBXMNq2FjCI2FKhdjScyJ4hxd++Ol M01P9FU/ =F3cq -----END PGP SIGNATURE----- [DECO-1115] Add local implementation for `dbutils.widgets` (databricks#93) * Added a new install group (`pip install 'databricks-sdk[notebook]'`). This allows us to safely pin ipywidgets for local installs. DBR can safely continue using `pip install databricks-sdk` or directly using the default build from master without conflicting dependencies. * OSS implementation of widgets is imported only on first use (possible only through OSS implementation of dbutils - `RemoteDbutils`). * Add a wrapper for ipywidgets to enable interactive widgets when in interactive **IPython** notebooks. https://user-images.githubusercontent.com/88345179/236443693-1c804107-ba21-4296-ba40-2b1e8e062d16.mov * Add default widgets implementation that returns a default value, when not in an interactive environment. https://user-images.githubusercontent.com/88345179/236443729-51185404-4d28-49c6-ade0-a665e154e092.mov  - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied Fix error message, ExportFormat -> ImportFormat (databricks#220) The proper argument is ImportFormat.AUTO, not ExportFormat.AUTO Correct the error message when `ImportFormat` is not provided to `workspace.upload`. Signed-off-by: Jessica Smith <[email protected]> Regenerate Python SDK using recent OpenAPI Specification (databricks#229) Spec commit sha: 17a3f7fe6 (7 July 2023) Breaking Changes: * Use CONSTANT_CASE for Enum constants. Many enums already use constant case in their definition, but some constants (like the SCIM Patch schema name) includes symbols `:` and numbers, so the SDK cannot use the enum value as the name. * Replace Query type with AlertQuery in sql.Alert class. * Removal of User.is_db_admin and User.profile_image_url. Changes: * Introduce CleanRooms API * Introduce TablesAPI.update() * Introduce Group.meta property * Fix SCIM Patch implementation * Introduce BaseRun.job_parameters and BaseRun.trigger_info * Introduce CreateJob.parameters * Fix spelling in file arrival trigger configuration * Introduce GitSource.job_source * Introduce RepairRun.rerun_dependent_tasks * Introduce Resolved*Values classes, RunIf, and RunJobTask * Introduce TaskNotificationSettings Later follow-up: * Names should split on Pascal-case word boundaries (see CloudProviderNodeStatus). This is an OpenAPI code gen change that needs to be made. Make workspace client also return runtime dbutils when in dbr (databricks#210) * `workspace_client.dbutils` always returned oss implementation of dbutils. We want it to also use dbr implementation when in dbr. * [x] Manually Test in dbr * [x] Test locally - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied Use .ConstantName defining target enum states for waiters (databricks#230) Uses of enums in generated code need to be updated to use `{{.ConstantName}}` instead of `{{.Content}}`. - [ ] `make test` run locally - [ ] `make fmt` applied - [ ] relevant integration tests applied Fix enum deserialization (databricks#234) In databricks#230, enums were changed so that enum field names did not necessarily match the enum value itself. However, the `_enum` helper method used during deserialization of a response containing an enum was not updated to handle this case. This PR corrects this method to check through the values of the `__members__` of an enum, as opposed to the keys.  - [ ] `make test` run locally - [ ] `make fmt` applied - [ ] relevant integration tests applied Fix enum deserialization, take 2 (databricks#235) We jumped the gun too quickly on databricks#234. This is the actual change which fixes the integration tests. - [x] The two failing integration tests (test_submitting_jobs and test_proxy_dbfs_mounts) both pass on this PR. Added toolchain configuration to `.codegen.json` (databricks#236) - Added toolchain config for automated releases - Added `CHANGELOG.md` template with OpenAPI SHA prep release changes Make OpenAPI spec location configurable (databricks#237) Introducing `DATABRICKS_OPENAPI_SPEC` environment variable to hold a filesystem location of `all-internal.json` spec. Rearrange imports in `databricks.sdk.runtime` to improve local editor experience (databricks#219)  The type hinting here means that VSCode does not give useful syntax highlights / code completions. This is the current experience on `main`: <img width="428" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/72d2c3eb-cc3a-4f95-9f09-7d43a8f2815e"> <img width="428" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/34b04de0-5996-4a2e-a0d2-b26a8c9d3da9"> With these changes this becomes: <img width="428" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/99a91d82-f06a-4883-b131-7f96a82edd80"> <img width="818" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/ce684fd0-f550-4afe-bc46-1187b6dd4b49">  - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied [DECO-1115] Add local implementation for `dbutils.widgets` (databricks#93) * Added a new install group (`pip install 'databricks-sdk[notebook]'`). This allows us to safely pin ipywidgets for local installs. DBR can safely continue using `pip install databricks-sdk` or directly using the default build from master without conflicting dependencies. * OSS implementation of widgets is imported only on first use (possible only through OSS implementation of dbutils - `RemoteDbutils`). * Add a wrapper for ipywidgets to enable interactive widgets when in interactive **IPython** notebooks. https://user-images.githubusercontent.com/88345179/236443693-1c804107-ba21-4296-ba40-2b1e8e062d16.mov * Add default widgets implementation that returns a default value, when not in an interactive environment. https://user-images.githubusercontent.com/88345179/236443729-51185404-4d28-49c6-ade0-a665e154e092.mov  - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied Fix error message, ExportFormat -> ImportFormat (databricks#220) The proper argument is ImportFormat.AUTO, not ExportFormat.AUTO Correct the error message when `ImportFormat` is not provided to `workspace.upload`. Signed-off-by: Jessica Smith <[email protected]> Regenerate Python SDK using recent OpenAPI Specification (databricks#229) Spec commit sha: 17a3f7fe6 (7 July 2023) Breaking Changes: * Use CONSTANT_CASE for Enum constants. Many enums already use constant case in their definition, but some constants (like the SCIM Patch schema name) includes symbols `:` and numbers, so the SDK cannot use the enum value as the name. * Replace Query type with AlertQuery in sql.Alert class. * Removal of User.is_db_admin and User.profile_image_url. Changes: * Introduce CleanRooms API * Introduce TablesAPI.update() * Introduce Group.meta property * Fix SCIM Patch implementation * Introduce BaseRun.job_parameters and BaseRun.trigger_info * Introduce CreateJob.parameters * Fix spelling in file arrival trigger configuration * Introduce GitSource.job_source * Introduce RepairRun.rerun_dependent_tasks * Introduce Resolved*Values classes, RunIf, and RunJobTask * Introduce TaskNotificationSettings Later follow-up: * Names should split on Pascal-case word boundaries (see CloudProviderNodeStatus). This is an OpenAPI code gen change that needs to be made. Make workspace client also return runtime dbutils when in dbr (databricks#210) * `workspace_client.dbutils` always returned oss implementation of dbutils. We want it to also use dbr implementation when in dbr. * [x] Manually Test in dbr * [x] Test locally - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied Use .ConstantName defining target enum states for waiters (databricks#230) Uses of enums in generated code need to be updated to use `{{.ConstantName}}` instead of `{{.Content}}`. - [ ] `make test` run locally - [ ] `make fmt` applied - [ ] relevant integration tests applied Fix enum deserialization, take 2 (databricks#235) We jumped the gun too quickly on databricks#234. This is the actual change which fixes the integration tests. - [x] The two failing integration tests (test_submitting_jobs and test_proxy_dbfs_mounts) both pass on this PR. Added toolchain configuration to `.codegen.json` (databricks#236) - Added toolchain config for automated releases - Added `CHANGELOG.md` template with OpenAPI SHA prep release changes Make OpenAPI spec location configurable (databricks#237) Introducing `DATABRICKS_OPENAPI_SPEC` environment variable to hold a filesystem location of `all-internal.json` spec. Rearrange imports in `databricks.sdk.runtime` to improve local editor experience (databricks#219)  The type hinting here means that VSCode does not give useful syntax highlights / code completions. This is the current experience on `main`: <img width="428" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/72d2c3eb-cc3a-4f95-9f09-7d43a8f2815e"> <img width="428" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/34b04de0-5996-4a2e-a0d2-b26a8c9d3da9"> With these changes this becomes: <img width="428" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/99a91d82-f06a-4883-b131-7f96a82edd80"> <img width="818" alt="image" src="https://github.com/databricks/databricks-sdk-py/assets/17158624/ce684fd0-f550-4afe-bc46-1187b6dd4b49">  - [x] `make test` run locally - [x] `make fmt` applied - [ ] relevant integration tests applied

* Add Issue Templates ([#208](#208)). * Fixed notebook native auth for jobs ([#209](#209)). * Replace `datatime.timedelta()` with `datetime.timedelta()` in codebase ([#207](#207)). * Support dod in python sdk ([#212](#212)). * [DECO-1115] Add local implementation for `dbutils.widgets` ([#93](#93)). * Fix error message, ExportFormat -> ImportFormat ([#220](#220)). * Regenerate Python SDK using recent OpenAPI Specification ([#229](#229)). * Make workspace client also return runtime dbutils when in dbr ([#210](#210)). * Use .ConstantName defining target enum states for waiters ([#230](#230)). * Fix enum deserialization ([#234](#234)). * Fix enum deserialization, take 2 ([#235](#235)). * Added toolchain configuration to `.codegen.json` ([#236](#236)). * Make OpenAPI spec location configurable ([#237](#237)). * Rearrange imports in `databricks.sdk.runtime` to improve local editor experience ([#219](#219)). API Changes: * Removed `maintenance()` method for [w.metastores](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/metastores.html) workspace-level service. * Added `enable_optimization()` method for [w.metastores](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/metastores.html) workspace-level service. * Added `update()` method for [w.tables](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/tables.html) workspace-level service. * Added `force` field for `databricks.sdk.service.catalog.DeleteAccountMetastoreRequest`. * Added `force` field for `databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`. * Removed `databricks.sdk.service.catalog.UpdateAutoMaintenance` dataclass. * Removed `databricks.sdk.service.catalog.UpdateAutoMaintenanceResponse` dataclass. * Added `databricks.sdk.service.catalog.UpdatePredictiveOptimization` dataclass. * Added `databricks.sdk.service.catalog.UpdatePredictiveOptimizationResponse` dataclass. * Added `databricks.sdk.service.catalog.UpdateTableRequest` dataclass. * Added `schema` field for `databricks.sdk.service.iam.PartialUpdate`. * Added `databricks.sdk.service.iam.PatchSchema` dataclass. * Added `trigger_info` field for `databricks.sdk.service.jobs.BaseRun`. * Added `health` field for `databricks.sdk.service.jobs.CreateJob`. * Added `job_source` field for `databricks.sdk.service.jobs.GitSource`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.JobEmailNotifications`. * Added `health` field for `databricks.sdk.service.jobs.JobSettings`. * Added `trigger_info` field for `databricks.sdk.service.jobs.Run`. * Added `run_job_output` field for `databricks.sdk.service.jobs.RunOutput`. * Added `run_job_task` field for `databricks.sdk.service.jobs.RunTask`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `health` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `notification_settings` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.Task`. * Added `run_job_task` field for `databricks.sdk.service.jobs.Task`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.TaskEmailNotifications`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.WebhookNotifications`. * Added `databricks.sdk.service.jobs.JobSource` dataclass. * Added `databricks.sdk.service.jobs.JobSourceDirtyState` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthMetric` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthOperator` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRule` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRules` dataclass. * Added `databricks.sdk.service.jobs.RunJobOutput` dataclass. * Added `databricks.sdk.service.jobs.RunJobTask` dataclass. * Added `databricks.sdk.service.jobs.TriggerInfo` dataclass. * Added `databricks.sdk.service.jobs.WebhookNotificationsOnDurationWarningThresholdExceededItem` dataclass. * Removed `whl` field for `databricks.sdk.service.pipelines.PipelineLibrary`. * Changed `delete_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `read_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `etag` field for `databricks.sdk.service.settings.DeletePersonalComputeSettingRequest` to be required. * Changed `etag` field for `databricks.sdk.service.settings.ReadPersonalComputeSettingRequest` to be required. * Added [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service. * Added `databricks.sdk.service.sharing.CentralCleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomAssetInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalog` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalogUpdate` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCollaboratorInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomNotebookInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomTableInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnMask` dataclass. * Added `databricks.sdk.service.sharing.ColumnTypeName` dataclass. * Added `databricks.sdk.service.sharing.CreateCleanRoom` dataclass. * Added `databricks.sdk.service.sharing.DeleteCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.GetCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.ListCleanRoomsResponse` dataclass. * Added `databricks.sdk.service.sharing.UpdateCleanRoom` dataclass. * Changed `query` field for `databricks.sdk.service.sql.Alert` to `databricks.sdk.service.sql.AlertQuery` dataclass. * Changed `value` field for `databricks.sdk.service.sql.AlertOptions` to `any` dataclass. * Removed `is_db_admin` field for `databricks.sdk.service.sql.User`. * Removed `profile_image_url` field for `databricks.sdk.service.sql.User`. * Added `databricks.sdk.service.sql.AlertQuery` dataclass. OpenAPI SHA: 36bb2292d778b9955eb3b799a39be94a83049b43, Date: 2023-07-18

* Add Issue Templates ([#208](#208)). * Fixed notebook native auth for jobs ([#209](#209)). * Replace `datatime.timedelta()` with `datetime.timedelta()` in codebase ([#207](#207)). * Support dod in python sdk ([#212](#212)). * [DECO-1115] Add local implementation for `dbutils.widgets` ([#93](#93)). * Fix error message, ExportFormat -> ImportFormat ([#220](#220)). * Regenerate Python SDK using recent OpenAPI Specification ([#229](#229)). * Make workspace client also return runtime dbutils when in dbr ([#210](#210)). * Use .ConstantName defining target enum states for waiters ([#230](#230)). * Fix enum deserialization ([#234](#234)). * Fix enum deserialization, take 2 ([#235](#235)). * Added toolchain configuration to `.codegen.json` ([#236](#236)). * Make OpenAPI spec location configurable ([#237](#237)). * Rearrange imports in `databricks.sdk.runtime` to improve local editor experience ([#219](#219)). * Updated account-level and workspace-level user management examples ([#241](#241)). API Changes: * Removed `maintenance()` method for [w.metastores](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/metastores.html) workspace-level service. * Added `enable_optimization()` method for [w.metastores](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/metastores.html) workspace-level service. * Added `update()` method for [w.tables](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/tables.html) workspace-level service. * Added `force` field for `databricks.sdk.service.catalog.DeleteAccountMetastoreRequest`. * Added `force` field for `databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`. * Removed `databricks.sdk.service.catalog.UpdateAutoMaintenance` dataclass. * Removed `databricks.sdk.service.catalog.UpdateAutoMaintenanceResponse` dataclass. * Added `databricks.sdk.service.catalog.UpdatePredictiveOptimization` dataclass. * Added `databricks.sdk.service.catalog.UpdatePredictiveOptimizationResponse` dataclass. * Added `databricks.sdk.service.catalog.UpdateTableRequest` dataclass. * Added `schema` field for `databricks.sdk.service.iam.PartialUpdate`. * Added `databricks.sdk.service.iam.PatchSchema` dataclass. * Added `trigger_info` field for `databricks.sdk.service.jobs.BaseRun`. * Added `health` field for `databricks.sdk.service.jobs.CreateJob`. * Added `job_source` field for `databricks.sdk.service.jobs.GitSource`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.JobEmailNotifications`. * Added `health` field for `databricks.sdk.service.jobs.JobSettings`. * Added `trigger_info` field for `databricks.sdk.service.jobs.Run`. * Added `run_job_output` field for `databricks.sdk.service.jobs.RunOutput`. * Added `run_job_task` field for `databricks.sdk.service.jobs.RunTask`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `health` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `notification_settings` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.Task`. * Added `run_job_task` field for `databricks.sdk.service.jobs.Task`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.TaskEmailNotifications`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.WebhookNotifications`. * Added `databricks.sdk.service.jobs.JobSource` dataclass. * Added `databricks.sdk.service.jobs.JobSourceDirtyState` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthMetric` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthOperator` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRule` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRules` dataclass. * Added `databricks.sdk.service.jobs.RunJobOutput` dataclass. * Added `databricks.sdk.service.jobs.RunJobTask` dataclass. * Added `databricks.sdk.service.jobs.TriggerInfo` dataclass. * Added `databricks.sdk.service.jobs.WebhookNotificationsOnDurationWarningThresholdExceededItem` dataclass. * Removed `whl` field for `databricks.sdk.service.pipelines.PipelineLibrary`. * Changed `delete_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `read_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `etag` field for `databricks.sdk.service.settings.DeletePersonalComputeSettingRequest` to be required. * Changed `etag` field for `databricks.sdk.service.settings.ReadPersonalComputeSettingRequest` to be required. * Added [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service. * Added `databricks.sdk.service.sharing.CentralCleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomAssetInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalog` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalogUpdate` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCollaboratorInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomNotebookInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomTableInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnMask` dataclass. * Added `databricks.sdk.service.sharing.ColumnTypeName` dataclass. * Added `databricks.sdk.service.sharing.CreateCleanRoom` dataclass. * Added `databricks.sdk.service.sharing.DeleteCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.GetCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.ListCleanRoomsResponse` dataclass. * Added `databricks.sdk.service.sharing.UpdateCleanRoom` dataclass. * Changed `query` field for `databricks.sdk.service.sql.Alert` to `databricks.sdk.service.sql.AlertQuery` dataclass. * Changed `value` field for `databricks.sdk.service.sql.AlertOptions` to `any` dataclass. * Removed `is_db_admin` field for `databricks.sdk.service.sql.User`. * Removed `profile_image_url` field for `databricks.sdk.service.sql.User`. * Added `databricks.sdk.service.sql.AlertQuery` dataclass. OpenAPI SHA: 0a1949ba96f71680dad30e06973eaae85b1307bb, Date: 2023-07-18

* Add Issue Templates ([#208](#208)). * Fixed notebook native auth for jobs ([#209](#209)). * Replace `datatime.timedelta()` with `datetime.timedelta()` in codebase ([#207](#207)). * Support dod in python sdk ([#212](#212)). * [DECO-1115] Add local implementation for `dbutils.widgets` ([#93](#93)). * Fix error message, ExportFormat -> ImportFormat ([#220](#220)). * Regenerate Python SDK using recent OpenAPI Specification ([#229](#229)). * Make workspace client also return runtime dbutils when in dbr ([#210](#210)). * Use .ConstantName defining target enum states for waiters ([#230](#230)). * Fix enum deserialization ([#234](#234)). * Fix enum deserialization, take 2 ([#235](#235)). * Added toolchain configuration to `.codegen.json` ([#236](#236)). * Make OpenAPI spec location configurable ([#237](#237)). * Rearrange imports in `databricks.sdk.runtime` to improve local editor experience ([#219](#219)). * Updated account-level and workspace-level user management examples ([#241](#241)). API Changes: * Removed `maintenance()` method for [w.metastores](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/metastores.html) workspace-level service. * Added `enable_optimization()` method for [w.metastores](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/metastores.html) workspace-level service. * Added `update()` method for [w.tables](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/tables.html) workspace-level service. * Added `force` field for `databricks.sdk.service.catalog.DeleteAccountMetastoreRequest`. * Added `force` field for `databricks.sdk.service.catalog.DeleteAccountStorageCredentialRequest`. * Removed `databricks.sdk.service.catalog.UpdateAutoMaintenance` dataclass. * Removed `databricks.sdk.service.catalog.UpdateAutoMaintenanceResponse` dataclass. * Added `databricks.sdk.service.catalog.UpdatePredictiveOptimization` dataclass. * Added `databricks.sdk.service.catalog.UpdatePredictiveOptimizationResponse` dataclass. * Added `databricks.sdk.service.catalog.UpdateTableRequest` dataclass. * Added `schema` field for `databricks.sdk.service.iam.PartialUpdate`. * Added `databricks.sdk.service.iam.PatchSchema` dataclass. * Added `trigger_info` field for `databricks.sdk.service.jobs.BaseRun`. * Added `health` field for `databricks.sdk.service.jobs.CreateJob`. * Added `job_source` field for `databricks.sdk.service.jobs.GitSource`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.JobEmailNotifications`. * Added `health` field for `databricks.sdk.service.jobs.JobSettings`. * Added `trigger_info` field for `databricks.sdk.service.jobs.Run`. * Added `run_job_output` field for `databricks.sdk.service.jobs.RunOutput`. * Added `run_job_task` field for `databricks.sdk.service.jobs.RunTask`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `health` field for `databricks.sdk.service.jobs.SubmitRun`. * Added `email_notifications` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `notification_settings` field for `databricks.sdk.service.jobs.SubmitTask`. * Added `health` field for `databricks.sdk.service.jobs.Task`. * Added `run_job_task` field for `databricks.sdk.service.jobs.Task`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.TaskEmailNotifications`. * Added `on_duration_warning_threshold_exceeded` field for `databricks.sdk.service.jobs.WebhookNotifications`. * Added `databricks.sdk.service.jobs.JobSource` dataclass. * Added `databricks.sdk.service.jobs.JobSourceDirtyState` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthMetric` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthOperator` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRule` dataclass. * Added `databricks.sdk.service.jobs.JobsHealthRules` dataclass. * Added `databricks.sdk.service.jobs.RunJobOutput` dataclass. * Added `databricks.sdk.service.jobs.RunJobTask` dataclass. * Added `databricks.sdk.service.jobs.TriggerInfo` dataclass. * Added `databricks.sdk.service.jobs.WebhookNotificationsOnDurationWarningThresholdExceededItem` dataclass. * Removed `whl` field for `databricks.sdk.service.pipelines.PipelineLibrary`. * Changed `delete_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `read_personal_compute_setting()` method for [a.account_settings](https://databricks-sdk-py.readthedocs.io/en/latest/account/account_settings.html) account-level service with new required argument order. * Changed `etag` field for `databricks.sdk.service.settings.DeletePersonalComputeSettingRequest` to be required. * Changed `etag` field for `databricks.sdk.service.settings.ReadPersonalComputeSettingRequest` to be required. * Added [w.clean_rooms](https://databricks-sdk-py.readthedocs.io/en/latest/workspace/clean_rooms.html) workspace-level service. * Added `databricks.sdk.service.sharing.CentralCleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomAssetInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalog` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCatalogUpdate` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomCollaboratorInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomNotebookInfo` dataclass. * Added `databricks.sdk.service.sharing.CleanRoomTableInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnInfo` dataclass. * Added `databricks.sdk.service.sharing.ColumnMask` dataclass. * Added `databricks.sdk.service.sharing.ColumnTypeName` dataclass. * Added `databricks.sdk.service.sharing.CreateCleanRoom` dataclass. * Added `databricks.sdk.service.sharing.DeleteCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.GetCleanRoomRequest` dataclass. * Added `databricks.sdk.service.sharing.ListCleanRoomsResponse` dataclass. * Added `databricks.sdk.service.sharing.UpdateCleanRoom` dataclass. * Changed `query` field for `databricks.sdk.service.sql.Alert` to `databricks.sdk.service.sql.AlertQuery` dataclass. * Changed `value` field for `databricks.sdk.service.sql.AlertOptions` to `any` dataclass. * Removed `is_db_admin` field for `databricks.sdk.service.sql.User`. * Removed `profile_image_url` field for `databricks.sdk.service.sql.User`. * Added `databricks.sdk.service.sql.AlertQuery` dataclass. OpenAPI SHA: 0a1949ba96f71680dad30e06973eaae85b1307bb, Date: 2023-07-18 [DECO-1115]: https://databricks.atlassian.net/browse/DECO-1115?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

kartikgupta-db added 5 commits May 4, 2023 09:21

wip

fb18854

Add IPyWidgets and DefaultWidgets

9f6e0a9

Merge branch 'main' of github.com:databricks/databricks-sdk-py into j…

f020963

…upyter-wdgets

revert .vscode/settings.json

f71fea7

lint

e330881

kartikgupta-db requested a review from nfx May 5, 2023 11:24

kartikgupta-db marked this pull request as draft May 5, 2023 11:24

kartikgupta-db added 2 commits May 5, 2023 17:11

test fix

38d486d

test fix

ed8b57a

nfx suggested changes May 5, 2023

View reviewed changes

xiaochen-db reviewed May 9, 2023

View reviewed changes

nfx reviewed May 10, 2023

View reviewed changes

databricks/sdk/_widgets/IPyWidgetUtils.py Show resolved Hide resolved

kartikgupta-db added 2 commits June 21, 2023 21:35

Merge branch 'main' of github.com:databricks/databricks-sdk-py into j…

b48ee60

…upyter-wdgets

refactor

ff43f1a

kartikgupta-db marked this pull request as ready for review June 21, 2023 22:27

kartikgupta-db requested review from nfx and xiaochen-db June 21, 2023 22:27

kartikgupta-db changed the title ~~Add local implementation for dbutils.widgets~~ [DECO-1115] Add local implementation for dbutils.widgets Jun 21, 2023

mgyucht self-requested a review June 28, 2023 12:21

mgyucht approved these changes Jun 28, 2023

View reviewed changes

kartikgupta-db added 3 commits June 29, 2023 00:21

Merge branch 'main' of github.com:databricks/databricks-sdk-py into j…

ab9787d

…upyter-wdgets

address feedback

f296be4

remove widget_names

9d6e017

mgyucht approved these changes Jun 29, 2023

View reviewed changes

xiaochen-db reviewed Jun 29, 2023

View reviewed changes

xiaochen-db approved these changes Jul 5, 2023

View reviewed changes

Merge branch 'main' into jupyter-wdgets

9a03c3e

kartikgupta-db merged commit 3a2d2e6 into main Jul 6, 2023

kartikgupta-db deleted the jupyter-wdgets branch July 6, 2023 02:06

nfx mentioned this pull request Jul 15, 2023

Release 0.2.0 #233

Closed

nfx mentioned this pull request Jul 17, 2023

Release 0.2.0 #238

Merged

Conversation

kartikgupta-db commented May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Uh oh!

kartikgupta-db commented May 5, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikgupta-db May 5, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikgupta-db Jun 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented Jun 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mgyucht left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikgupta-db Jun 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kartikgupta-db Jun 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kartikgupta-db commented May 5, 2023 •

edited

Loading

kartikgupta-db May 5, 2023 •

edited

Loading

kartikgupta-db Jun 28, 2023 •

edited

Loading

codecov-commenter commented Jun 21, 2023 •

edited

Loading

kartikgupta-db Jun 28, 2023 •

edited

Loading

kartikgupta-db Jun 28, 2023 •

edited

Loading