ZEPPELIN-3517 Remove zeppelin.pyspark.python in PySparkInterpreter#2993
ZEPPELIN-3517 Remove zeppelin.pyspark.python in PySparkInterpreter#2993zjffdu wants to merge 1 commit intoapache:masterfrom
Conversation
|
LGTM |
felixcheung
left a comment
felixcheung
left a comment
There was a problem hiding this comment.
why not just map it to PYSPARK_DRIVER_PYTHON or PYSPARK_PYTHON
|
@felixcheung The only place to do mapping is in zeppelin server side, and I it is better not to put any interpreter specific login in zeppelin server side. And I think it's better for zeppelin to use the same configuration as spark instead of introducing zeppelin specific configuration. |
|
I’m all for not having special config though I’d prefer not to the break users with existing note or config right away, without some sort of deprecation period. But it’s your call
|
|
Thanks @felixcheung . I still think it is better to remove it. As it is difficult to deprecate this and let users to aware of that. Zeppelin is a notebook instead of library. If we don't remove it and just deprecate it in documentation, I believe most of users won't be know that and would continue to use it and may ask question in user mail list why they hit python version mismatch issue between driver and executors. |
### What is this PR for? `zeppelin.pyspark.python` should be removed as it is zeppelin specific property, and only affect the driver, but not on executor. So we should use spark property instead. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3517 ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <[email protected]> Closes apache#2993 from zjffdu/ZEPPELIN-3517 and squashes the following commits: 24dafa1 [Jeff Zhang] ZEPPELIN-3517. Remove zeppelin.pyspark.python in PySparkInterpreter
3 participants
What is this PR for?
zeppelin.pyspark.pythonshould be removed as it is zeppelin specific property, and only affect the driver, but not on executor. So we should use spark property instead.What type of PR is it?
[Bug Fix]
Todos
What is the Jira issue?
How should this be tested?
Screenshots (if appropriate)
Questions: