Skip to content

ZEPPELIN-598 ] Dynamic loading for Interpreter#652

Closed
cloverhearts wants to merge 2 commits intoapache:masterfrom
cloverhearts:ZEPPELIN-598
Closed

ZEPPELIN-598 ] Dynamic loading for Interpreter#652
cloverhearts wants to merge 2 commits intoapache:masterfrom
cloverhearts:ZEPPELIN-598

Conversation

@cloverhearts
Copy link
Copy Markdown
Member

What is this PR for?

Use of external libraries that are included in the Roadmap,
The interpreter needs related to dynamic loading capabilities.

_It has been re-written from the first newly created PR._
#631

What type of PR is it?

Feature

Todos

Is there a relevant Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-598

How should this be tested?

By annotating the interpreter setting items below, try using the loadDynamicInterpreter method.
Or create a new interpreter loads created in the local Maven repository.
incubator-zeppelin/zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java

    ZEPPELIN_INTERPRETERS("zeppelin.interpreters", "org.apache.zeppelin.spark.SparkInterpreter,"
        + "org.apache.zeppelin.spark.PySparkInterpreter,"
        + "org.apache.zeppelin.spark.SparkSqlInterpreter,"
        + "org.apache.zeppelin.spark.DepInterpreter,"
        + "org.apache.zeppelin.markdown.Markdown,"
...

Questions:

  • Does the licenses files need update? no
  • Is there breaking changes for older versions? no
  • Does this needs documentation? no

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also add this property to conf/zeppelin-env.sh.template, conf/zeppelin-site.xml.template and docs/install/install.md ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Leemoonsoo Thank you for feed back.
I will add that information.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the name ZEPPELIN_INTERPRETER_REPO_DIR and zeppelin.user.interpreter.dir are little bit confusing.

How about more intuitive name such as ZEPPELIN_INTERPRETER_DOWNLOAD_DIR and zeppelin.interpreter.download.dir ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's the temp directory for downloaded interpreter, why not making it more general like: ZEPPELIN_PLUGINS_DOWNLOAD_DIR?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corneadoug because Interpreter is loaded by conf/interpreter.sh and other pluggable component are not, copying dependencies in a directory is only requirement of conf/interpreter.sh. So this configuration shouldn't be generalized for other pluggable module.

Dependencies for other pluggable components will directly loaded from local repository for dependency loader, which is configured by ZEPPELIN_DEP_LOCALREPO.

@Leemoonsoo
Copy link
Copy Markdown
Member

Is it closed on purpose?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants