Verify metastore exists in current workspace by pritishpai · Pull Request #735 · databrickslabs/ucx

pritishpai · 2023-12-23T00:19:31Z

Resolved #104

codecov · 2023-12-23T00:20:39Z

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (bba54e5) 80.18% compared to head (317808d) 80.46%.
Report is 1 commits behind head on main.

❗ Current head 317808d differs from pull request most recent head 199e7fa. Consider uploading reports for the commit 199e7fa to get more accurate results

Files	Patch %	Lines
src/databricks/labs/ucx/runtime.py	25.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #735      +/-   ##
==========================================
+ Coverage   80.18%   80.46%   +0.27%     
==========================================
  Files          44       44              
  Lines        4628     4658      +30     
  Branches      856      858       +2     
==========================================
+ Hits         3711     3748      +37     
+ Misses        694      687       -7     
  Partials      223      223

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

nfx · 2023-12-29T09:02:47Z

+
+class MetastoreNotFoundError(Exception):
+    def __init__(self, message="Metastore not found in the workspace"):
+        self.message = message


It's not necessary - exceptions can be turned to strings easily

nfx · 2023-12-29T09:04:22Z

+            raise MetastoreNotFoundError()
+        return True
+
+    def check_metastore_existence(self) -> bool:


It's not used

Remove this method, it increases the surface area, that we don't need now

It is used in the verify_metastore method and also unit tested (if that is what you meant by surface area)

it's not used anywhere else

Removing it with the fix for failing integration tests which was because of not handling the non-UC env permissions error.

nfx · 2023-12-29T19:11:35Z

+            raise MetastoreNotFoundError()
+        return True
+
+    def check_metastore_existence(self) -> bool:


it's not used anywhere else

Adding a command to verify that a metastore exists in the workspace

* Added assessment step to estimate the size of DBFS root tables ([#741](#741)). * Added `TableMapping` functionality to table migrate ([#752](#752)). * Added `databricks labs ucx move` command to move tables and schemas between catalogs ([#756](#756)). * Added functionality to determine migration method based on DBFS Root ([#759](#759)). * Added `get_tables_to_migrate` functionality in the mapping module ([#755](#755)). * Added retry and rate limit to rename workspace group operation and corrected rate limit for reflecting account groups to workspace ([#751](#751)). * Adopted `databricks-labs-blueprint` library for common utilities to be reused in the other projects ([#758](#758)). * Converted `RuntimeBackend` query executions exceptions to SDK exceptions ([#769](#769)). * Fixed issue with missing users and temp groups after workspace-local groups migration and skip table when crawling table size if it does not exist anymore ([#770](#770)). * Improved error handling by not failing group rename step if a group was removed from account before reflecting it to workspace ([#762](#762)). * Improved error message inference from failed workflow runs ([#753](#753)). * Moved `TablesMigrate` to a separate module ([#747](#747)). * Reorganized assessment dashboard to increase readability ([#738](#738)). * Updated databricks-sdk requirement from ~=0.16.0 to ~=0.17.0 ([#773](#773)). * Verify metastore exists in current workspace ([#735](#735)).

pritishpai requested review from a team and dmoore247 December 23, 2023 00:19

pritishpai had a problem deploying to account-admin December 23, 2023 00:19 — with GitHub Actions Failure

pritishpai force-pushed the fix/issue_104 branch from 13bb770 to 2a2a1e7 Compare December 23, 2023 00:23

pritishpai had a problem deploying to account-admin December 23, 2023 00:23 — with GitHub Actions Failure

nfx requested changes Dec 23, 2023

View reviewed changes

Comment thread src/databricks/labs/ucx/cli.py Outdated

pritishpai force-pushed the fix/issue_104 branch from 58e9124 to d09fd77 Compare December 28, 2023 20:46

pritishpai temporarily deployed to account-admin December 28, 2023 20:46 — with GitHub Actions Inactive

pritishpai had a problem deploying to account-admin December 28, 2023 21:15 — with GitHub Actions Failure

pritishpai changed the title ~~Added new cli command to verify metastore in current workspace~~ Verify metastore exists in current workspace Dec 28, 2023

pritishpai had a problem deploying to account-admin December 28, 2023 21:31 — with GitHub Actions Failure

pritishpai temporarily deployed to account-admin December 28, 2023 21:56 — with GitHub Actions Inactive

pritishpai had a problem deploying to account-admin December 29, 2023 03:29 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin December 29, 2023 03:59 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin December 29, 2023 04:42 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin December 29, 2023 04:51 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin December 29, 2023 05:38 — with GitHub Actions Failure

nfx requested changes Dec 29, 2023

View reviewed changes

pritishpai had a problem deploying to account-admin December 29, 2023 18:01 — with GitHub Actions Failure

nfx approved these changes Dec 29, 2023

View reviewed changes

pritishpai had a problem deploying to account-admin December 29, 2023 19:22 — with GitHub Actions Failure

pritishpai force-pushed the fix/issue_104 branch from f5d58ef to df3122e Compare January 3, 2024 22:35

pritishpai had a problem deploying to account-admin January 3, 2024 22:35 — with GitHub Actions Error

pritishpai had a problem deploying to account-admin January 3, 2024 22:39 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin January 3, 2024 22:43 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin January 4, 2024 03:54 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin January 4, 2024 04:45 — with GitHub Actions Failure

pritishpai temporarily deployed to account-admin January 4, 2024 15:01 — with GitHub Actions Inactive

Verify metastore cli command

8b199f1

Adding a command to verify that a metastore exists in the workspace

pritishpai added 10 commits January 4, 2024 12:43

mypy changes

e93ce1e

Added unit tests

9868435

New class for VerifyHasMetastore and unit tests

afa741e

Fixing rebase leftover code

add45ae

Adding verify metastore calls

244c1ea

Addressing PR comments #1

d3594f7

Fixing integration errors

0ae2500

PermissionDenied error and tests

db9f4d5

Fixing linting errors

967e5de

Fixing error message

317808d

pritishpai force-pushed the fix/issue_104 branch from 582aefe to 317808d Compare January 4, 2024 17:44

pritishpai temporarily deployed to account-admin January 4, 2024 17:44 — with GitHub Actions Inactive

nfx reviewed Jan 5, 2024

View reviewed changes

Comment thread src/databricks/labs/ucx/workspace_access/verification.py Outdated

remove debug logging

199e7fa

nfx had a problem deploying to account-admin January 5, 2024 09:41 — with GitHub Actions Failure

nfx merged commit f6819f2 into main Jan 5, 2024

nfx deleted the fix/issue_104 branch January 5, 2024 09:42

nfx mentioned this pull request Jan 12, 2024

Release v0.9.0 #776

Merged

FastLee pushed a commit that referenced this pull request Jan 19, 2024

Verify metastore exists in current workspace (#735)

45e3dde

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Verify metastore exists in current workspace#735

Verify metastore exists in current workspace#735
nfx merged 12 commits intomainfrom
fix/issue_104

pritishpai commented Dec 23, 2023

Uh oh!

codecov bot commented Dec 23, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

nfx Dec 29, 2023

Uh oh!

nfx Dec 29, 2023

Uh oh!

pritishpai Dec 29, 2023

Uh oh!

nfx Dec 29, 2023

Uh oh!

pritishpai Dec 29, 2023

Uh oh!

Uh oh!

nfx Dec 29, 2023

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pritishpai commented Dec 23, 2023

Uh oh!

codecov bot commented Dec 23, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

nfx Dec 29, 2023

Choose a reason for hiding this comment

Uh oh!

nfx Dec 29, 2023

Choose a reason for hiding this comment

Uh oh!

pritishpai Dec 29, 2023

Choose a reason for hiding this comment

Uh oh!

nfx Dec 29, 2023

Choose a reason for hiding this comment

Uh oh!

pritishpai Dec 29, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nfx Dec 29, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Dec 23, 2023 •

edited

Loading