Skip to content

Commit 563ef3a

Browse files
committed
Review feedback
1 parent 6ad06d4 commit 563ef3a

File tree

2 files changed

+8
-4
lines changed

2 files changed

+8
-4
lines changed

core/src/main/scala/org/apache/spark/SparkContext.scala

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1027,7 +1027,7 @@ class SparkContext(
10271027
* The SparkContext object contains a number of implicit conversions and parameters for use with
10281028
* various Spark features.
10291029
*/
1030-
object SparkContext {
1030+
object SparkContext extends Logging {
10311031

10321032
private[spark] val SPARK_JOB_DESCRIPTION = "spark.job.description"
10331033

@@ -1246,6 +1246,10 @@ object SparkContext {
12461246
scheduler
12471247

12481248
case "yarn-standalone" | "yarn-cluster" =>
1249+
if (master == "yarn-standalone") {
1250+
logWarning(
1251+
"\"yarn-standalone\" is deprecated as of Spark 1.0. Use \"yarn-cluster\" instead.")
1252+
}
12491253
val scheduler = try {
12501254
val clazz = Class.forName("org.apache.spark.scheduler.cluster.YarnClusterScheduler")
12511255
val cons = clazz.getConstructor(classOf[SparkContext])

docs/running-on-yarn.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -90,9 +90,9 @@ For example:
9090
--worker-memory 2g \
9191
--worker-cores 1
9292

93-
The above starts a YARN client program which starts the default Application Master. Then SparkPi will be run as a child thread of Application Master. The client will periodically poll the Application Master for status updates and display them in the console. The client will exit once your application has finished running.
93+
The above starts a YARN client program which starts the default Application Master. Then SparkPi will be run as a child thread of Application Master. The client will periodically poll the Application Master for status updates and display them in the console. The client will exit once your application has finished running. Refer to the "Viewing Logs" section below for how to see driver and executor logs.
9494

95-
Because the application is run on a remote machine where the Application Master is running, applications that involve local interaction, such as spark-shell, will not work well.
95+
Because the application is run on a remote machine where the Application Master is running, applications that involve local interaction, such as spark-shell, will not work.
9696

9797
## Launching a Spark application with yarn-client mode.
9898

@@ -136,7 +136,7 @@ When log aggregation isn't turned on, logs are retained locally on each machine
136136

137137
See [Building Spark with Maven](building-with-maven.html) for instructions on how to build Spark using Maven.
138138

139-
# Important Notes
139+
# Important notes
140140

141141
- Before Hadoop 2.2, YARN does not support cores in container resource requests. Thus, when running against an earlier version, the numbers of cores given via command line arguments cannot be passed to YARN. Whether core requests are honored in scheduling decisions depends on which scheduler is in use and how it is configured.
142142
- The local directories used by Spark executors will be the local directories configured for YARN (Hadoop YARN config yarn.nodemanager.local-dirs). If the user specifies spark.local.dir, it will be ignored.

0 commit comments

Comments
 (0)