Fixed `databricks labs ucx repair-run` command to execute correctly by prajin-29 · Pull Request #801 · databrickslabs/ucx

prajin-29 · 2024-01-17T07:07:03Z

Changes

Fixing the issue for repair run CLI databricks labs ucx repair-run . When a CLI tries to repair run a job before if updates its response json to either FAILED or SUCCESS it was failing with NoneType exception.

Added a check in repair_run inside install.py to check the status of the response and wait for 20 seconds to get it updated .

Enhanced the code to repair run already repaired job.

Linked issues

closes #787

Resolves #787

Functionality

modified the cli command databricks labs ucx repair-run which was failing in regression testing

Tests

This has been manually tested
added unit tests test_repair_run_result_state in test_install.py
This was tested using integration test

codecov · 2024-01-17T07:08:34Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (af80620) 84.07% compared to head (add2404) 84.13%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #801      +/-   ##
==========================================
+ Coverage   84.07%   84.13%   +0.05%     
==========================================
  Files          39       39              
  Lines        4872     4890      +18     
  Branches      913      916       +3     
==========================================
+ Hits         4096     4114      +18     
  Misses        564      564              
  Partials      212      212

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

nfx

Rewrite to retried decorator.

nfx · 2024-01-17T08:49:31Z

src/databricks/labs/ucx/install.py

+
+            while not state.result_state and (time.time() - start_time < timeout):
+                logger.info("Waiting for the result_state to update the state")
+                time.sleep(10)


This is not unit testable, see how we use retried() decorator in workspace access package (dbsql permissions, secrets acls, etc).

@nfx .Updated the code with retried logic.

nfx · 2024-01-17T08:51:48Z

src/databricks/labs/ucx/install.py

            latest_job_run = job_runs[0]
            state = latest_job_run.state
+
+            while not state.result_state and (time.time() - start_time < timeout):


Refactor this into private method and decode with retried

@nfx .Refactored the same with retried logic.

nfx · 2024-01-18T10:24:43Z

src/databricks/labs/ucx/install.py

+    def _get_result_state(self, job_id):
+        job_runs = list(self._ws.jobs.list_runs(job_id=job_id, limit=1))
+        latest_job_run = job_runs[0]
+        if not latest_job_run.state.result_state:
+            logger.info("Waiting for the result_state to update the state")
+            time.sleep(10)
+        job_state = latest_job_run.state.result_state.value
+        return job_state


Suggested change

def _get_result_state(self, job_id):

job_runs = list(self._ws.jobs.list_runs(job_id=job_id, limit=1))

latest_job_run = job_runs[0]

if not latest_job_run.state.result_state:

logger.info("Waiting for the result_state to update the state")

time.sleep(10)

job_state = latest_job_run.state.result_state.value

return job_state

def _get_result_state(self, job_id):

job_runs = list(self._ws.jobs.list_runs(job_id=job_id, limit=1))

if len(job_runs) == 0:

raise AttributeError("no job runs found")

latest_job_run = job_runs[0]

if not latest_job_run.state.result_state:

raise AttributeError("no result state in job run")

job_state = latest_job_run.state.result_state.value

return job_state

you have retried(on=[AttributeError], but don't throw it anywhere

If latest_job_run.state is None then latest_job_run.state.result_state.value will throw AttributeError. But I have rewritten now to raise the exception.

For Job Runs during the initial stage itself we are exiting immediately if don't have any job run for the job_id with proper message.

# Conflicts: # src/databricks/labs/ucx/install.py

* Added `databricks labs ucx validate-groups-membership` command to validate groups to see if they have same membership across acount and workspace level ([#772](#772)). * Added baseline for getting Azure Resource Role Assignments ([#764](#764)). * Added issue and pull request templates ([#791](#791)). * Added linked issues to PR template ([#793](#793)). * Added optional `debug_truncate_bytes` parameter to the config and extend the default log truncation limit ([#782](#782)). * Added support for crawling grants and applying Hive Metastore UDF ACLs ([#812](#812)). * Changed Python requirement from 3.10.6 to 3.10 ([#805](#805)). * Extend error handling of delta issues in crawlers and hive metastore ([#795](#795)). * Fixed `databricks labs ucx repair-run` command to execute correctly ([#801](#801)). * Fixed handling of `DELTASHARING` table format ([#802](#802)). * Fixed listing of workflows via CLI ([#811](#811)). * Fixed logger import path for DEBUG notebook ([#792](#792)). * Fixed move table command to delete table/view regardless if permissions are present, skipping corrupted tables when crawling table size and making existing tests more stable ([#777](#777)). * Fixed the issue of `databricks labs ucx installations` and `databricks labs ucx manual-workspace-info` ([#814](#814)). * Increase the unit test coverage for cli.py ([#800](#800)). * Mount Point crawler lists /Volume with four variations which is confusing ([#779](#779)). * Updated README.md to remove mention of deprecated install.sh ([#781](#781)). * Updated `bug` issue template ([#797](#797)). * Fixed writing log readme in multiprocess safe way ([#794](#794)).

Fixing the bug for the repair run

e3b8f0c

prajin-29 requested review from a team and nsenno-dbr January 17, 2024 07:07

prajin-29 temporarily deployed to account-admin January 17, 2024 07:07 — with GitHub Actions Inactive

Fixing the bug for the repair run

772399a

prajin-29 temporarily deployed to account-admin January 17, 2024 07:10 — with GitHub Actions Inactive

nfx requested changes Jan 17, 2024

View reviewed changes

prajin-29 added 3 commits January 18, 2024 09:51

Fixing the bug for the repair run

b518dfe

Merge branch 'main' into feature/fix_repai_run

1ecc4c9

Fixing the bug for the repair run

dd50f3c

prajin-29 had a problem deploying to account-admin January 18, 2024 07:43 — with GitHub Actions Failure

Fixing the bug for the repair run

3a5e93d

prajin-29 had a problem deploying to account-admin January 18, 2024 07:48 — with GitHub Actions Failure

Fixing the bug for the repair run

657c02f

prajin-29 temporarily deployed to account-admin January 18, 2024 09:51 — with GitHub Actions Inactive

prajin-29 requested a review from nfx January 18, 2024 10:08

nfx requested changes Jan 18, 2024

View reviewed changes

Fixing the bug for the repair run

9bf2d5f

prajin-29 had a problem deploying to account-admin January 18, 2024 11:14 — with GitHub Actions Failure

Fixing the bug for the repair run

8d7e119

prajin-29 had a problem deploying to account-admin January 18, 2024 13:15 — with GitHub Actions Failure

Merge branch 'main' into feature/fix_repai_run

735d648

# Conflicts: # src/databricks/labs/ucx/install.py

prajin-29 had a problem deploying to account-admin January 18, 2024 15:01 — with GitHub Actions Failure

Fixing the bug for the repair run

add2404

prajin-29 had a problem deploying to account-admin January 18, 2024 15:04 — with GitHub Actions Failure

prajin-29 had a problem deploying to account-admin January 19, 2024 03:46 — with GitHub Actions Failure

prajin-29 had a problem deploying to account-admin January 19, 2024 04:05 — with GitHub Actions Failure

prajin-29 temporarily deployed to account-admin January 19, 2024 06:31 — with GitHub Actions Inactive

nfx changed the title ~~Fixing the Issue for Repair Run databricks labs ucx repair-run~~ Fixed databricks labs ucx repair-run command to execute correctly Jan 19, 2024

nfx approved these changes Jan 19, 2024

View reviewed changes

nfx merged commit b45fa41 into main Jan 19, 2024

nfx deleted the feature/fix_repai_run branch January 19, 2024 09:14

nfx mentioned this pull request Jan 19, 2024

Release v0.10.0 #819

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed `databricks labs ucx repair-run` command to execute correctly#801

Fixed `databricks labs ucx repair-run` command to execute correctly#801
nfx merged 11 commits intomainfrom
feature/fix_repai_run

prajin-29 commented Jan 17, 2024

Uh oh!

codecov bot commented Jan 17, 2024 •

edited

Loading

Uh oh!

nfx left a comment

Uh oh!

nfx Jan 17, 2024

Uh oh!

prajin-29 Jan 18, 2024

Uh oh!

nfx Jan 17, 2024

Uh oh!

prajin-29 Jan 18, 2024

Uh oh!

nfx Jan 18, 2024 •

edited

Loading

Uh oh!

nfx Jan 18, 2024

Uh oh!

prajin-29 Jan 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

prajin-29 commented Jan 17, 2024

Changes

Linked issues

Functionality

Tests

Uh oh!

codecov bot commented Jan 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

nfx left a comment

Choose a reason for hiding this comment

Uh oh!

nfx Jan 17, 2024

Choose a reason for hiding this comment

Uh oh!

prajin-29 Jan 18, 2024

Choose a reason for hiding this comment

Uh oh!

nfx Jan 17, 2024

Choose a reason for hiding this comment

Uh oh!

prajin-29 Jan 18, 2024

Choose a reason for hiding this comment

Uh oh!

nfx Jan 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nfx Jan 18, 2024

Choose a reason for hiding this comment

Uh oh!

prajin-29 Jan 18, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Jan 17, 2024 •

edited

Loading

nfx Jan 18, 2024 •

edited

Loading