ZEPPELIN-598 ] Dynamic loading for Interpreter#631
Closed
cloverhearts wants to merge 17 commits intoapache:masterfrom
cloverhearts:ZEPPELIN-598-dynamic
Closed
ZEPPELIN-598 ] Dynamic loading for Interpreter#631cloverhearts wants to merge 17 commits intoapache:masterfrom cloverhearts:ZEPPELIN-598-dynamic
cloverhearts wants to merge 17 commits intoapache:masterfrom
cloverhearts:ZEPPELIN-598-dynamic
Conversation
Member
There was a problem hiding this comment.
Can you please change indention here and in all cases below back to the original, to make sure it is consistent with the rest of the project code?
The reason for that is, other than being inconsistent, such changes make reviews much harder.
Here is the relevant style guide section javaguide.html#s4.6.3-horizontal-alignment
Author: agura <[email protected]> Closes #624 from agura/zeppelin-596 and squashes the following commits: f93a2d5 [agura] ZEPPELIN-596 Use default marshaller in Ignite interpreter by default
Member
There was a problem hiding this comment.
Guava's Joiner or String.format() might come handy here and below.
Member
|
Looks good to me, except for major style\formatting nit above. |
Member
Author
|
@bzz Thank you for feedback! |
### What is this PR for? After the Shiro PR was merged, the Navbar status was changed to include the token status/name, however that part is reserved to show the Websocket status. Because of that it usually show a green circle with Disconnected next to it. This PR sets the status to Websocket only. ### What type of PR is it? Bug Fix ### Is there a relevant Jira issue? No ### How should this be tested? You shouldn't see see Disconnected next to a green round in the navbar anymore ### Screenshots (if appropriate) Bug: <img width="386" alt="screen shot 2016-01-13 at 12 02 24 pm" src="https://cloud.githubusercontent.com/assets/6766992/12283801/927d05e8-b9ed-11e5-8911-9725b4a7c94a.png"> ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Sejun Ra <[email protected]> Closes #628 from sejunra/fix/NavbarConnectedStatus and squashes the following commits: ee8a2fd [Sejun Ra] Restore Navbar connection status for websocket only
### What is this PR for? This PR is to fix the `z.select()` defaultValue ### What type of PR is it? Bug Fix ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-581 ### How should this be tested? If you run: ``` val selected = z.select("formName", "option1", Seq(("option1", "option1DisplayName"), ("option2", "option2DisplayName"))) println(selected) ``` you should see: `res27: Object = option1` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Damien CORNEAU <[email protected]> Closes #623 from corneadoug/fix/selectDisplayFormDefaultValue and squashes the following commits: 15b4199 [Damien CORNEAU] Pass defaultValue parameter to z.select
Member
There was a problem hiding this comment.
### What is this PR for? hayssams Leemoonsoo Just tested #586 with a local LDAP, works well for me. Included a sample config. ### What type of PR is it? Documentation / Sample testing. ### How should this be tested? Change configs to point to a local/available LDAP, uncomment the parameters, replacing the individual values. Author: Rohit Choudhary <[email protected]> Closes #625 from rconline/SHIRO_LDAP and squashes the following commits: fa0b69c [Rohit Choudhary] Enabling annonymous access, commenting out AuthBasic. ead38fd [Rohit Choudhary] Tested with local LDAP configurations.
### What is this PR for? #625 removed `[urls]` from conf/shiro.ini and that cause following exception on start up ``` ERROR [2016-01-14 20:46:38,646] ({main} EnvironmentLoader.java[initEnvironment]:146) - Shiro environment initialization failed org.apache.shiro.config.ConfigurationException: Unable to instantiate class [anon] for object named '/**'. Please ensure you've specified the fully qualified class name correctly. at org.apache.shiro.config.ReflectionBuilder.createNewInstance(ReflectionBuilder.java:151) at org.apache.shiro.config.ReflectionBuilder.buildObjects(ReflectionBuilder.java:119) at org.apache.shiro.config.IniSecurityManagerFactory.buildInstances(IniSecurityManagerFactory.java:161) at org.apache.shiro.config.IniSecurityManagerFactory.createSecurityManager(IniSecurityManagerFactory.java:124) at org.apache.shiro.config.IniSecurityManagerFactory.createSecurityManager(IniSecurityManagerFactory.java:102) at org.apache.shiro.config.IniSecurityManagerFactory.createInstance(IniSecurityManagerFactory.java:88) at org.apache.shiro.config.IniSecurityManagerFactory.createInstance(IniSecurityManagerFactory.java:46) at org.apache.shiro.config.IniFactorySupport.createInstance(IniFactorySupport.java:123) at org.apache.shiro.util.AbstractFactory.getInstance(AbstractFactory.java:47) at org.apache.shiro.web.env.IniWebEnvironment.createWebSecurityManager(IniWebEnvironment.java:203) at org.apache.shiro.web.env.IniWebEnvironment.configure(IniWebEnvironment.java:99) at org.apache.shiro.web.env.IniWebEnvironment.init(IniWebEnvironment.java:92) at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:45) at org.apache.shiro.util.LifecycleUtils.init(LifecycleUtils.java:40) at org.apache.shiro.web.env.EnvironmentLoader.createEnvironment(EnvironmentLoader.java:221) at org.apache.shiro.web.env.EnvironmentLoader.initEnvironment(EnvironmentLoader.java:133) at org.apache.shiro.web.env.EnvironmentLoaderListener.contextInitialized(EnvironmentLoaderListener.java:58) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:782) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:424) at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:774) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:249) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:717) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at org.eclipse.jetty.server.handler.HandlerCollection.doStart(HandlerCollection.java:229) at org.eclipse.jetty.server.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:172) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:95) at org.eclipse.jetty.server.Server.doStart(Server.java:282) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64) at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:114) Caused by: org.apache.shiro.util.UnknownClassException: Unable to load class named [anon] from the thread context, current, or system/application ClassLoaders. All heuristics have been exhausted. Class could not be found. at org.apache.shiro.util.ClassUtils.forName(ClassUtils.java:148) at org.apache.shiro.util.ClassUtils.newInstance(ClassUtils.java:164) at org.apache.shiro.config.ReflectionBuilder.createNewInstance(ReflectionBuilder.java:144) ... 29 more ``` ### What type of PR is it? Hot Fix ### Todos * [x] - fix ### Is there a relevant Jira issue? no ### How should this be tested? just build and start zeppelin with default configuration ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <[email protected]> Closes #639 from Leemoonsoo/hotfix_shiro and squashes the following commits: 7a75c48 [Lee moon soo] Hotfix shiro.ini
### What is this PR for? improving pom.xml ### What type of PR is it Improvement ### Todos * [x] - Update pom.xml ### Is there a relevant Jira issue? May not require issue ### How should this be tested? I use ```mvn clean package -Pspark-1.4 -Phadoop-2.6 -DskipTests``` can cause the module Phoenix error and I think this should not appear. Reason about this issue is sqlline:sqlline:jar:1.1.8 is served from conjar repo. But sqlline:sqlline:jar:1.1.9 is served from maven repo. So add the dependency of sqlline1.1.9 can solve it. Of course, you can fix this issue by another way, but this way may be in common use. http://mvnrepository.com/artifact/sqlline/sqlline/1.1.9 ### Screenshots (if appropriate) Before `BUILD FAILURE` When you add this dependency After ``` [INFO] Zeppelin: Apache Phoenix Interpreter ............... SUCCESS [ 2.497 s] [INFO] Zeppelin: PostgreSQL interpreter ................... SUCCESS [ 0.572 s] [INFO] Zeppelin: Tajo interpreter ......................... SUCCESS [ 0.557 s] [INFO] Zeppelin: Flink .................................... SUCCESS [ 4.695 s] [INFO] Zeppelin: Apache Ignite interpreter ................ SUCCESS [ 0.536 s] [INFO] Zeppelin: Kylin interpreter ........................ SUCCESS [ 0.520 s] [INFO] Zeppelin: Lens interpreter ......................... SUCCESS [ 1.592 s] [INFO] Zeppelin: Cassandra ................................ SUCCESS [ 35.948 s] [INFO] Zeppelin: Elasticsearch interpreter ................ SUCCESS [ 1.323 s] [INFO] Zeppelin: web Application .......................... SUCCESS [ 52.998 s] [INFO] Zeppelin: Server ................................... SUCCESS [ 11.377 s] [INFO] Zeppelin: Packaging distribution ................... SUCCESS [ 1.054 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:03 min [INFO] Finished at: 2016-01-14T14:53:26+08:00 [INFO] Final Memory: 160M/1060M [INFO] ------------------------------------------------------------------------ ``` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: zhangminglei <[email protected]> Closes #637 from hash-X/patch-2 and squashes the following commits: 40f2eca [zhangminglei] Update pom.xml
### What is this PR for?
Improving logging information on error
### What type of PR is it?
Fix/Improvement
### Todos
* [x] - fix logging
### Is there a relevant Jira issue?
May not require issue
### How should this be tested?
Can be tested with any error for storage initialization
For example:
1. in `conf/zeppelin-env.sh` add the following line
```
export ZEPPELIN_NOTEBOOK_STORAGE="org.apache.zeppelin.notebook.repo.VFSNotebookRepo,org.apache.zeppelin.notebook.repo.DummyNotebookRepo"
```
2. Start Zeppelin
3. Previously you would see one line warning, now you would see more detailed trace-back as well.
### Screenshots (if appropriate)
Before:
```
Failed to initialize com.nflabs.zeppelinhub.notebook.repo.DummyNotebookRepo notebook storage class java.lang.reflect.InvocationTargetException
```
After:
```
WARN [2016-01-13 22:43:05,557] ({main} NotebookRepoSync.java[<init>]:81) - Failed to initialize com.nflabs.zeppelinhub.notebook.repo.DummyNotebookRepo notebook storage class
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Khalid Huseynov <[email protected]>
Closes #634 from khalidhuseynov/fix-logging and squashes the following commits:
dd65555 [Khalid Huseynov] fix logging to informative one
### What is this PR for?
A lot of time travis fails with
Tests run: 11, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.77 sec <<< FAILURE! - in org.apache.zeppelin.notebook.NotebookTest
testSelectingReplImplementation(org.apache.zeppelin.notebook.NotebookTest) Time elapsed: 0.025 sec <<< ERROR!
org.apache.zeppelin.interpreter.InterpreterException: mock2 interpreter not found
at org.apache.zeppelin.notebook.NoteInterpreterLoader.get(NoteInterpreterLoader.java:148)
at org.apache.zeppelin.notebook.Note.run(Note.java:365)
at org.apache.zeppelin.notebook.NotebookTest.testSelectingReplImplementation(NotebookTest.java:119)
.
.
.
.
Tests in error:
NotebookTest.testSelectingReplImplementation:119 » Interpreter mock2 interpret...
This is fix for that error, this doesn't happens always. But here is a lot when this failed last time. https://s3.amazonaws.com/archive.travis-ci.org/jobs/102172625/log.txt
### What type of PR is it?
Improvement
### Todos
* [x] - Clear registered interpreters after a test case is executed.
### Is there a relevant Jira issue?
N/A
Author: Prabhjyot Singh <[email protected]>
Closes #642 from prabhjyotsingh/travisBuildFix and squashes the following commits:
ddda3c5 [Prabhjyot Singh] registeredInterpreters.clear() in tearDown
b07bca7 [Prabhjyot Singh] instead of 2 create 1 directory
### What is this PR for? add a keyboard shortcut to add paragraph above to keep difference with previous add paragraph below ### What type of PR is it? Improvement ### Todos ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-611 ### How should this be tested? click into a paragraph and press ctrl+alt+n ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Michael Chen <[email protected]> Closes #643 from MikeTYChen/ZEPPELIN-611 and squashes the following commits: 732917f [Michael Chen] change to a e9e419d [Michael Chen] keyboard shortcut to add above
### What is this PR for? A few sentences describing the overall goals of the pull request's commits. ### What type of PR is it? Bug Fix ### Todos * [x] - Have error output in multiline ### Is there a relevant Jira issue? N/A ### How should this be tested? Run something that will output in error, for example try running "asdf" with spark interpreter ### Screenshots (if appropriate) Before: <img width="1439" alt="screen shot 2016-01-15 at 12 52 24 pm" src="https://cloud.githubusercontent.com/assets/674497/12349221/f7ca1f4c-bb93-11e5-8fc9-04c06abf62ea.png"> After: <img width="1429" alt="screen shot 2016-01-15 at 12 52 10 pm" src="https://cloud.githubusercontent.com/assets/674497/12349220/f7c834ca-bb93-11e5-8823-e316152f64b4.png"> Author: Prabhjyot Singh <[email protected]> Closes #641 from prabhjyotsingh/multiLineError and squashes the following commits: 867a3e0 [Prabhjyot Singh] small change to have error in multiline
### What is this PR for? It would be better if Zeppelin provides looking good documentation. ### What type of PR is it? Improvement ### Todos * [x] - Fix typos, grammars and Increase readability writingzeppelininterpreter.md * [x] - Also spark.md * [x] - tutorial.md * [x] - cassandra.md * [x] - elasticsearch.md * [x] - flink.md * [x] - hive.md * [x] - lens.md * [x] - markdown.md * [x] - rest-interpreter.md ### Is there a relevant Jira issue? No : ) ### How should this be tested? These documentations are in [here](http://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/). ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Ryu Ah young <[email protected]> Closes #578 from AhyoungRyu/Fix-Typo and squashes the following commits: 925f30e [Ryu Ah young] Fix the list of 'Configure your Interpreter' 9b768e9 [Ryu Ah young] Change .md -> .html in interpreter.md 785dcbb [Ryu Ah young] Fix odd space in spark.md a5d26d7 [Ryu Ah young] Fix version 1.5.1 -> 1.5.2 in spark.md ef77027 [Ryu Ah young] Fix Upper case -> Lower case in cassandra.md 0918f79 [Ryu Ah young] Fix grammar error and Increase readability interpreters.md f3d0173 [Ryu Ah young] Add numbering for dividing the table list and Fix grammar error rest-interpreter.md fe4dfff [Ryu Ah young] Fix image size unit px -> % and Increase readability markdown.md 11b32db [Ryu Ah young] Remove useless 'to' in lens.md 3eec4cf [Ryu Ah young] Remove numbering flink.md aac2e01 [Ryu Ah young] Add short description about Apache hive and Increase readability hive.md d08092a [Ryu Ah young] Add highlight for scala code and Increase readability flink.md 8690e02 [Ryu Ah young] Increase readability and remove useless <hr> tag elasticsearch.md 9c008cf [Ryu Ah young] Increase readability and Remove useless <br>, <hr> tag cassandra.md 3041b95 [Ryu Ah young] Change rebuild -> rebuilding in spark.md 99b1e9e [Ryu Ah young] Remove unnecessary space in writingzeppelininterpreter.md c63e59f [Ryu Ah young] Increase readability and delete useless <br> tag tutorial.md e867df3 [Ryu Ah young] Increase readability and delete useless <br>, <hr> tag spark.md f72b1cb [Ryu Ah young] Add .sh next to .bin/zeppein-daemon 466f82c [Ryu Ah young] Fix some typos and grammars in writingzeppelininterpreter.md
### What is this PR for? spark/pom.xml has some duplicated entry with spark-dependencies/pom.xml. this pr removes them. ### What type of PR is it? Refactoring ### Is there a relevant Jira issue? no ### How should this be tested? N/A ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <[email protected]> Closes #647 from Leemoonsoo/move_spark_profile and squashes the following commits: be0dbe7 [Lee moon soo] remove duplicated entries from spark/pom.xml
…m maven repository ### What is this PR for? This PR is related to #590. This documentation may explain the overall process of **Dynamic Interpreter Loading** in [Helium Proposal](https://issues.apache.org/jira/browse/ZEPPELIN-533). Moreover, at the last this documentation, Zeppelin users can get the basic information about step by step of interpreter setting & configuring & binding. Since I assumed that they are novice at Zeppelin. ### What type of PR is it? Documentation ### Todos * [x] - Add docs image under `/docs/assets/themes/zeppelin/img/docs-img/` * [x] - Add dynamicinterpreter.md * [x] - Add dynamicinterpreter.html location to `docs/_includes/themes/zeppelin/_navigation.html` ### Is there a relevant Jira issue? 1. [ZEPPELIN-533 Helium](https://issues.apache.org/jira/browse/ZEPPELIN-533) 2. [ZEPPELIN-546](https://issues.apache.org/jira/browse/ZEPPELIN-546) ### How should this be tested? I add a link of this documentation to `interpreter tab` in Zeppelin web page.  ### Screenshots (if appropriate) Hopefully, below image helps you to understand this process : )  ### Questions: * Does the licenses files need update? No. * Is there breaking changes for older versions? No. * Does this needs documentation? No. Author: Ryu Ah young <[email protected]> Closes #609 from AhyoungRyu/ZEPPELIN-546-docs and squashes the following commits: d0675e0 [Ryu Ah young] ZEPPELIN-546-docs: Fix some sentences 266dac0 [Ryu Ah young] ZEPPELIN-546-docs: Add content for licence e180a8e [Ryu Ah young] ZEPPELIN-546-docs: Fix typo again d3cc50f [Ryu Ah young] ZEPPELIN-546-docs: Fix typo a9ec7d7 [Ryu Ah young] ZEPPELIN-546-docs: Add dynamicinterpreterload.html location to _navigation.html 16b46f4 [Ryu Ah young] ZEPPELIN-546-docs: Add dynamicinterpreterload.md 3067025 [Ryu Ah young] ZEPPELIN-546-docs: Add docs image
Only you need to add to the classpath the jdbc connector jar and the interpreter add the particular properties for each db.
In the file zeppelin-daemon.sh add:
ZEPPELIN_CLASSPATH+=":${ZEPPELIN_HOME}/jdbc/jdbc/connector jar"
Author: beeva-victorgarcia <[email protected]>
Author: Victor <[email protected]>
Author: vgmartinez <[email protected]>
Closes #361 from vgmartinez/jdbc_interpreter and squashes the following commits:
2513c1b [vgmartinez] Merge branch 'master' into jdbc_interpreter
8046692 [vgmartinez] merged with master
e602621 [beeva-victorgarcia] remove spaces
37a4c1a [beeva-victorgarcia] remove dependency
4085849 [beeva-victorgarcia] rebase branch
bd20ac2 [beeva-victorgarcia] remove README.md
2f93406 [beeva-victorgarcia] clean commit
f0ad06d [beeva-victorgarcia] Merge branch 'master' of https://github.com/apache/incubator-zeppelin
a0e0d54 [beeva-victorgarcia] add some test
fe92f89 [beeva-victorgarcia] add multiple connections for interpreter
a06718c [beeva-victorgarcia] -a
09006f1 [Victor] fix test
462c3b1 [Victor] Merge branch 'master' of https://github.com/apache/incubator-zeppelin into jdbc_interpreter
a66a5b7 [Victor] add descriptions
710699c [Victor] deleted cassandra/.cache-main from commit
4f28f5a [Victor] change psql to jdbc
53d0a81 [Victor] generic interpreter for jdbc
2 tasks
Member
Author
|
Thank you everyone. |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this PR for?
Use of external libraries that are included in the Roadmap,
The interpreter needs related to dynamic loading capabilities.
What type of PR is it?
Feature
Todos
Is there a relevant Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-598
How should this be tested?
By annotating the interpreter setting items below, try using the loadDynamicInterpreter method.
Or create a new interpreter loads created in the local Maven repository.
incubator-zeppelin/zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
Questions: