Description
For the 1.0 release we should have a basic history server for non-standalone modes (e.g. Yarn, Mesos). It can just be initialized with a storage directory and watch for new applications. Ideally this should be able to respect UI security filters as well.
This might require us to add some sort of "COMPLETED" file or some synchronization to know when the UI has finished running so we don't try to load the file a bunch of times.
Attachments
Issue Links
- is blocked by
-
SPARK-1132 Persisting Web UI through refactoring the SparkListener interface
-
- Resolved
-
- is related to
-
SPARK-1408 Modify Spark on Yarn to point to the history server when app finishes
-
- Resolved
-