Skip to content

Creates cluster policy (ucx-policy) to be used by all UCX compute#853

Merged
nfx merged 34 commits intomainfrom
feature/clusterpolicy
Feb 9, 2024
Merged

Creates cluster policy (ucx-policy) to be used by all UCX compute#853
nfx merged 34 commits intomainfrom
feature/clusterpolicy

Conversation

@HariGS-DB
Copy link
Copy Markdown
Contributor

@HariGS-DB HariGS-DB commented Jan 29, 2024 •
edited
Loading

Changes

Creates a policy which is used by all compute runs of UCX

Linked issues

Resolves #694

Functionality

  • added relevant user documentation
  • added new CLI command
  • modified existing command: databricks labs ucx ...
  • added a new workflow
  • modified existing workflow: ...
  • added a new table
  • modified existing table: ...

Tests

  • manually tested
  • added unit tests
  • added integration tests
  • verified on staging environment (screenshot attached)

@HariGS-DB HariGS-DB added the pr/do-not-merge this pull request is not ready to merge label Jan 29, 2024
nfx
nfx requested changes Jan 29, 2024
View reviewed changes
@HariGS-DB HariGS-DB marked this pull request as ready for review January 31, 2024 22:06
@HariGS-DB HariGS-DB requested review from a team and nkvuong January 31, 2024 22:06
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 31, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (baf3984) 86.55% compared to head (d1e8806) 87.09%.

❗ Current head d1e8806 differs from pull request most recent head 1d3ebbb. Consider uploading reports for the commit 1d3ebbb to get more accurate results

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #853      +/-   ##
==========================================
+ Coverage   86.55%   87.09%   +0.53%     
==========================================
  Files          41       41              
  Lines        5162     5161       -1     
  Branches      938      931       -7     
==========================================
+ Hits         4468     4495      +27     
+ Misses        481      460      -21     
+ Partials      213      206       -7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@HariGS-DB HariGS-DB removed the request for review from nkvuong January 31, 2024 22:14
nfx
nfx requested changes Feb 1, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rebase and apply comments

@HariGS-DB
Copy link
Copy Markdown
Contributor Author

HariGS-DB commented Feb 7, 2024

@nfx Can we merge this?

nfx
nfx approved these changes Feb 9, 2024
View reviewed changes
Copy link
Copy Markdown
Collaborator

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@nfx nfx merged commit b94df54 into main Feb 9, 2024
@nfx nfx deleted the feature/clusterpolicy branch February 9, 2024 09:49
This was referenced Feb 9, 2024
Closed
Closed
nfx added a commit that referenced this pull request Feb 9, 2024
@nfx
Release v0.12.0
92025c0 
* Added CLI Command `databricks labs ucx save-uc-compatible-roles` ([#863](#863)).
* Added dashboard widget with table count by storage and format ([#852](#852)).
* Added verification of group permissions ([#841](#841)).
* Checking pipeline cluster config and cluster policy in 'crawl_pipelines' task ([#864](#864)).
* Created cluster policy (ucx-policy) to be used by all UCX compute. This may require customers to reinstall UCX. ([#853](#853)).
* Skip scanning objects that were removed on platform side since the last scan time, so that integration tests are less flaky ([#922](#922)).
* Updated assessment documentation ([#873](#873)).

Dependency updates:

 * Updated databricks-sdk requirement from ~=0.18.0 to ~=0.19.0 ([#930](#930)).
@nfx nfx mentioned this pull request Feb 9, 2024
Merged
nfx added a commit that referenced this pull request Feb 9, 2024
@nfx
Release v0.12.0 (#931)
158ae61 
* Added CLI Command `databricks labs ucx save-uc-compatible-roles`
([#863](#863)).
* Added dashboard widget with table count by storage and format
([#852](#852)).
* Added verification of group permissions
([#841](#841)).
* Checking pipeline cluster config and cluster policy in
'crawl_pipelines' task
([#864](#864)).
* Created cluster policy (ucx-policy) to be used by all UCX compute.
This may require customers to reinstall UCX.
([#853](#853)).
* Skip scanning objects that were removed on platform side since the
last scan time, so that integration tests are less flaky
([#922](#922)).
* Updated assessment documentation
([#873](#873)).

Dependency updates:

* Updated databricks-sdk requirement from ~=0.18.0 to ~=0.19.0
([#930](#930)).
@nfx nfx mentioned this pull request Mar 7, 2024
Closed
1 task
dmoore247 pushed a commit that referenced this pull request Mar 23, 2024
@HariGS-DB @dmoore247
Creates cluster policy (ucx-policy) to be used by all UCX compute (#853)
09e905a
dmoore247 pushed a commit that referenced this pull request Mar 23, 2024
@nfx @dmoore247
Release v0.12.0 (#931)
8ce8edf 
* Added CLI Command `databricks labs ucx save-uc-compatible-roles`
([#863](#863)).
* Added dashboard widget with table count by storage and format
([#852](#852)).
* Added verification of group permissions
([#841](#841)).
* Checking pipeline cluster config and cluster policy in
'crawl_pipelines' task
([#864](#864)).
* Created cluster policy (ucx-policy) to be used by all UCX compute.
This may require customers to reinstall UCX.
([#853](#853)).
* Skip scanning objects that were removed on platform side since the
last scan time, so that integration tests are less flaky
([#922](#922)).
* Updated assessment documentation
([#873](#873)).

Dependency updates:

* Updated databricks-sdk requirement from ~=0.18.0 to ~=0.19.0
([#930](#930)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nfx nfx nfx approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Create cluster policy on installation and use that one for all of the compute

2 participants

@HariGS-DB @nfx