Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19577

insert into a partition datasource table with InMemoryCatalog after the partition location alter by alter command failed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.2.0
    • None
    • SQL

    Description

      If we use InMemoryCatalog, then we insert into a partition datasource table, which partition location has changed by `alter table t partition(a="xx") set location $newpath`, the insert operation is ok, and the data can be insert into $newpath, while if we then select partition from the table, it will not return the value we inserted.

      The reason is that the InMemoryFileIndex to inferPartition by the table's rootPath, it does not track the user specific $newPath which provided by alter command.

      Attachments

        Activity

          People

            Unassigned Unassigned
            windpiger Song Jun
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: