Skip to content

[BUG] DROP WITH PURGE tasks are not guaranteed to complete #269

@eric-maynard

Description

@eric-maynard

Is this a possible security vulnerability?

  • This is NOT a possible security vulnerability

Describe the bug

When dropTable is called with purgeRequested, the Polaris server will create Tasks in order to delete data files associated with the table being dropped. However, if the service dies before the tasks are completed they may not resume gracefully on startup. Some of that table's data files may not be cleaned up.

To Reproduce

Call dropTable with purgeRequested, and kill the Polaris service before the relevant Tasks complete.

Actual Behavior

Tasks are not resumed on startup

Expected Behavior

Tasks associated with a purge should reliably complete, or the user should be made aware that they cannot be completed

Additional context

I've met with @collado-mike and others to discuss a long-term solution here, but it may take some time. In the interim, we discussed some workarounds and stopgaps.

System information

n/a

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions