Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
2.1.0
Description
val spark = SparkSession
.builder
.appName("PartitionDropTest")
.master("local[2]").enableHiveSupport()
.getOrCreate()
val sentenceData = spark.createDataFrame(Seq(
(0, "a"),
(1, "b"),
(2, "c")))
.toDF("id", "name")
spark.sql("drop table if exists licllocal.partition_table")
sentenceData.write.mode(SaveMode.Overwrite).partitionBy("id").saveAsTable("licllocal.partition_table")
spark.sql("alter table licllocal.partition_table drop partition(id='')")
spark.table("licllocal.partition_table").show()
the result is
|name| id| +----+---+ +----+---+
Maybe the partition match have something wrong when the partition value is set to empty string