Skip to content

Implementing force delete for catalog and schema#61

Merged
haogang merged 10 commits intounitycatalog:mainfrom
ravivj-db:handle-deletes
Jul 9, 2024
Merged

Implementing force delete for catalog and schema#61
haogang merged 10 commits intounitycatalog:mainfrom
ravivj-db:handle-deletes

Conversation

@ravivj-db
Copy link
Collaborator

@ravivj-db ravivj-db commented Jun 18, 2024

PR Checklist

  • A description of the changes is added to the description of this PR.
  • If there is a related issue, make sure it is linked to this PR.
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added or modified a feature, documentation in docs is updated

Description of changes

  • Proper Implementation of delete in schema and catalog taking into account the force flag.
  • If force flag is set to true, all child entities are deleted in the same transaction.
  • If force flag is set to false, and child entities are present, error is thrown

Related Bug

Solve the linked issue

Testing

Test cases for the above mentioned scenarios have been added and are running fine.

@haogang haogang self-requested a review June 26, 2024 17:04
.replace("{full_name}", ApiClient.urlEncode(fullName.toString()));

localVarRequestBuilder.uri(URI.create(memberVarBaseUri + localVarPath));
List<Pair> localVarQueryParams = new ArrayList<>();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move the URL parsing logic to a generic util function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is autogenerated code. Currently we are not changing the client code unless there's an error in the logic.

try {
CatalogInfoDAO catalogInfo = getCatalogDAO(session, name);
if (catalogInfo != null) {
List<SchemaInfoDAO> schemas = schemaRepository.listSchemas(session, catalogInfo.getId(), Optional.of(100));
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is 100 the page size or the max result? The logic is wrong if it's the max result

It's inefficient to list all schemas when force is not specified. Can we list one schema first to check, and only list others when force is specified?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you are correct. 100 currently stands for pageSize. We should leave it empty in order to get all the existing child entities.
I have implemented the similar logic for all entities (level-2, level-3) now :

  • First check if a single child entity exists.
  • if not force throw exception
  • else get a list of all child entities of that type by setting maxResults as empty/null and delete all of them.

CatalogInfoDAO catalogInfo = getCatalogDAO(session, name);
if (catalogInfo != null) {
List<SchemaInfoDAO> schemas = schemaRepository.listSchemas(session, catalogInfo.getId(), Optional.of(100));
if (schemas != null & !schemas.isEmpty()) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

&& ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, good catch.

if (schemaInfo != null) {

// handle tables
List<TableInfoDAO> tables = tableRepository.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to unify the handling for different entity types so that we don't miss new entities types.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no field currently in the db schema to indicate if an entity is level-3 entity (for e.g. table, function, volume). Also there is one different singleton instance of repository associated with each type(i.e.TableRepository, FunctionRepository, VolumeRepository are all different classes and we maintain singletons of each). Given this structure, I am not sure what would be a good way to unify the handling of all entity types.

@ravivj-db ravivj-db requested a review from haogang July 6, 2024 12:46
@haogang haogang merged commit 84e3cac into unitycatalog:main Jul 9, 2024
dennyglee pushed a commit that referenced this pull request Sep 2, 2024
* delete schema functionality

* prettier
tdas pushed a commit that referenced this pull request Sep 5, 2024
* delete schema functionality

* prettier
rtyler pushed a commit to rtyler/unitycatalog that referenced this pull request Sep 5, 2024
* delete schema functionality

* prettier
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants