You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit bcf4862
Browse filesBrowse the repository at this point in the historyBrowse files
fix: apply the captured configs to the Celeborn frame size and capture DEBUG logs in tests
The native Celeborn writer read spark.comet.shuffle.rss.maxFrameBytes in
getWriter, before the writer applies the captured configs. Carry the
configs on NativeShuffleSpec, which getWriter can reach through the
dependency, and apply them to that read as well as in the writer, in
place of CometNativeShuffleInputRDD and its iterator.
Spark's LogAppender keeps INFO and above by default, so set its threshold
to DEBUG for the memory pool line the tests read.
0 commit comments