Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17388

Support for inferring type date/timestamp/decimal for partition column

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • SQL
    • None

    Description

      Currently, Spark only supports to infer IntegerType, LongType, DoubleType and StringType.

      DecimalType is being tried but it seems it never infers type as DecimalType as DoubleType is being tried first.

      Also, DateType and TimestampType can be inferred. It seems it is a pretty common to use both for a partition column.

      It'd be great if they can be inferred as both rather than just StringType.

      Attachments

        Issue Links

          Activity

            People

              gurwls223 Hyukjin Kwon
              gurwls223 Hyukjin Kwon
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: