[SPARK-20957][SS][TESTS] Fix o.a.s.sql.streaming.StreamingQueryManagerSuite listing

zsxwing · tdas · commit bc537e40ade0 · 2017-06-05T14:34:10.000-07:00
## What changes were proposed in this pull request? When stopping StreamingQuery, StreamExecution will set `streamDeathCause` then notify StreamingQueryManager to remove this query. So it's possible that when `q2.exception.isDefined` returns `true`, StreamingQueryManager's active list still has `q2`. This PR just puts the checks into `eventually` to fix the flaky test. ## How was this patch tested? Jenkins Author: Shixiong Zhu <shixiong@databricks.com> Closes #18180 from zsxwing/SPARK-20957.
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala
@@ -78,9 +78,9 @@ class StreamingQueryManagerSuite extends StreamTest with BeforeAndAfter {
       eventually(Timeout(streamingTimeout)) {
         require(!q2.isActive)
         require(q2.exception.isDefined)
+        assert(spark.streams.get(q2.id) === null)
+        assert(spark.streams.active.toSet === Set(q3))
       }
-      assert(spark.streams.get(q2.id) === null)
-      assert(spark.streams.active.toSet === Set(q3))
     }
   }
 

Original file line number	Diff line number	Diff line change
`@@ -78,9 +78,9 @@ class StreamingQueryManagerSuite extends StreamTest with BeforeAndAfter {`
`78`	`78`	`eventually(Timeout(streamingTimeout)) {`
`79`	`79`	`require(!q2.isActive)`
`80`	`80`	`require(q2.exception.isDefined)`
	`81`	`+ assert(spark.streams.get(q2.id) === null)`
	`82`	`+ assert(spark.streams.active.toSet === Set(q3))`
`81`	`83`	`}`
`82`		`- assert(spark.streams.get(q2.id) === null)`
`83`		`- assert(spark.streams.active.toSet === Set(q3))`
`84`	`84`	`}`
`85`	`85`	`}`
`86`	`86`