You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d334434
Browse filesBrowse the repository at this point in the historyBrowse files
docs: qualify the memoryOverhead recommendation so it cannot shrink the container
Setting spark.executor.memoryOverhead replaces Spark's derived value rather
than adding to it (ResourceProfile.calculateOverHeadMemory uses getOrElse), so
a flat 2g recommendation reduces the allowance for executors large enough that
the factor already derives more than that. Describe the derivation accurately,
including the 0.4 factor for non-JVM apps on Kubernetes in cluster mode and the
configurable floor on Spark 4.0+, scope 2g to small executors, and point larger
deployments at spark.executor.memoryOverheadFactor.
0 commit comments