Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-13047

Pyspark Params.hasParam should not throw an error

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.6.1, 2.0.0
    • ML, PySpark
    • None

    Description

      Pyspark Params class has a method hasParam(paramName) which returns True if the class has a parameter by that name, but throws an AttributeError otherwise. There is not currently a way of getting a Boolean to indicate if a class has a parameter. With Spark 2.0 we could modify the existing behavior of hasParam or add an additional method with this functionality.

      In Python:

      from pyspark.ml.classification import NaiveBayes
      nb = NaiveBayes(smoothing=0.5)
      print nb.hasParam("smoothing")
      print nb.hasParam("notAParam")
      

      produces:
      > True
      > AttributeError: 'NaiveBayes' object has no attribute 'notAParam'

      However, in Scala:

      import org.apache.spark.ml.classification.NaiveBayes
      val nb  = new NaiveBayes()
      nb.hasParam("smoothing")
      nb.hasParam("notAParam")
      

      produces:
      > true
      > false

      Attachments

        Activity

          People

            sethah Seth Hendrickson
            sethah Seth Hendrickson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: