Skip to content

R and SparkR Support [WIP]#702

Closed
echarles wants to merge 85 commits intoapache:masterfrom
datalayer-attic:rscala-z
Closed

R and SparkR Support [WIP]#702
echarles wants to merge 85 commits intoapache:masterfrom
datalayer-attic:rscala-z

Conversation

@echarles
Copy link
Copy Markdown
Member

@echarles echarles commented Feb 7, 2016

What is this PR for?

Implement R and SpakR Intepreter as part of the Spark Interpreter Group. It also implements R and Scala binding (in both directions).

What type of PR is it?

[Feature]

Todos

  • - Documentation
  • - Unit test (if relevant, as we depend on R being available on the host)
  • - Assess licensing (a priori ok as we don't delive anything out of ASL2, but we should corectly phrase the NOTICE as we depend on non-ASL2 comptabile licenses (R) to run the interpreter).

Is there a relevant Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-156 SparkR support

How should this be tested?

You need R available on the host running the notebook.

For Centos: yum install R R-devel
For Ubuntu: apt-get install r-base r-cran-rserve

Install additional R packages:

curl https://cran.r-project.org/src/contrib/Archive/rscala/rscala_1.0.6.tar.gz -o /tmp/rscala_1.0.6.tar.gz
R CMD INSTALL /tmp/rscala_1.0.6.tar.gz
R -e "install.packages('ggplot2', repos = 'http://cran.us.r-project.org')"
R -e install.packages('knitr', repos = 'http://cran.us.r-project.org')
  • Build + launch Zeppelin and test the R note.

!!! you need rscala_1.0.6 (if not, you need to build with -Drscala.version=...)

Screenshots (if appropriate)

Simple R

Simple R

Plot

Plot

Scala R Binding

Scala R Binding

R Scala Binding

R Scala Binding

SparkR

SparkR

Questions:

  • Does the licenses files need update? to be checked... (cfr R needs to be available to make this interpreter operational).
  • Is there breaking changes for older versions? No
  • Does this needs documentation? Yes

@echarles
Copy link
Copy Markdown
Member Author

echarles commented Feb 7, 2016

CLI fails for profiles 1.3, 1.2 and 1.1 with object r is not a member of package org.apache.spark.api which is normal as SparkR was introduced in 1.4.

Profiles 1.4, 1.5 and 1.6 are green.

@Leemoonsoo
Copy link
Copy Markdown
Member

Thanks for the contribution.

I think spark/lib/rscala_2.10-1.0.6.jar shouldn't be included in the source tree. Can this be downloaded on build time?

And rscala_2.10-1.0.6.jar looks like available on BSD 3 Clause license according to here. Then, you'll need add license of the library into zeppelin-distribution/src/bin_license/LICENSE, and add LICENSE file under zeppelin-distribution/src/bin_license/licenses directory.

Most interpreters include at least one unittest or integration test. So I suggest have an test.

I think R interpreter related source file need to be excluded for profile spark-1.3, spark-1.2, spark-1.1, so CI doesn't fail for those profiles. I think maven capable of doing it.

@ghost
Copy link
Copy Markdown

ghost commented Feb 8, 2016

I guess this isn't part of the 72-hour moratorium...

@Leemoonsoo

Rscala Actually its not just that rscala can be downloaded at build time. It will be downloaded whenever the user updates their R package library. R data scientists users do quite regularly because the functionality is built into R. There is no way to prevent an updated rscala from being used in place of the rscala included with this code. (The issue is the R-side of the rscala library more than the java side, but that's beside the point.)

This has already created breakages: The author of rscala recently fixed a bug that affected Windows users. The new version broke Eric's implementation, which had to be refactored.

Licenses Regarding licenses --you are correct that rscala is available under BSD. I convinced the author to switch from GPL to BSD so it could be used in PR 208.

Tests You are astute to require unit tests. The reason this code is passing CI is that it isn't being tested at all. Neither R, nor the classes here, are ever executed.

Related to that: There's no code here to handle changes in the SparkInterpreter class, which has caused severe breakages in the spark-interpreter/R integration in the past. There's also no code in here to handle the common Zeppelin/Spark misconfigurations that are 99% of the user issues that arise when you add R to zeppelin; or to handle spark-related issues at all.

One reason for this is, surely, because Eric only added the integration a few weeks ago, after I pointed-out on the mailing list that the functionality was absent. Since Eric based his code on mine (which he is absolutely entitled to do), I'm pretty sure it will start to have the same issues if it ever develops a user base.

Other There are a slew of other issues here related to integration with the rest of Zeppelin, presentation of R in a manner that R-using data scientists will be familiar with, and allowing the full R functionality.

As just one example: it doesn't support R base-plotting, and I'm pretty sure it won't support interactive visualizations either.

I am not going to list all of those issues here now, when we're trying to resolve the issue of PR 208 -- which does handle all of them, and quite a few more. This is because I worked through them over the last six months in responding to requests and issues from users.

(Just to be clear, this is @elbamos -- I've been switching from OS X to Ubuntu, and it looks like I logged in with the wrong e-mail address and a git account I made an eternity ago and never used.)

@hsaputra
Copy link
Copy Markdown

hsaputra commented Feb 8, 2016

@echarles Looks like this PR was trying to just add support for SparkR package.
AFAIK this can be addressed #208 PR, can't it?

@echarles
Copy link
Copy Markdown
Member Author

echarles commented Feb 8, 2016

@Leemoonsoo

  • rscala is now downloaded at build time
  • bin_license updated
  • Unit test added
  • spark-1.3, spark-1.2, spark-1.1 should now build with maven machinery

@justsomeone1001

  • This PR support visualization.
  • I have added a very first inistial documentation to help the setup.

@hsaputra

  • Once you get SparR, you have R (just like PySpark, you get Python)
  • I have opened this PR to take into acocunt with my comments on R Interpreter for Zeppelin #208 PR (see [1] and [2])

[1] http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-dev/201512.mbox/%3C5683E471.9010001%40apache.org%3E
[2] #208 (comment)

Thx for your comments so far and looking what Travis will tell...

@ghost
Copy link
Copy Markdown

ghost commented Feb 8, 2016

@echarles It doesn't support base plotting. Do you have an example of code to show that it supports interactive visualizations? I believe it does not support them.

Regarding @hsaputra 's comment -- I think your comment on 208, was that you felt the functionality that is in PR 208 but not in this PR, isn't necessary.

@smartinsightsfromdata
Copy link
Copy Markdown

@echarles @justsomeone1001 could you kindly clarify if native visualization refers to Zeppelin visualisations or R?

Visualisation is most relevant around data exploration and analytics results sharing (where notebooks make most sense). I think for R and Python users it would be good to know which visualisation will work out-of-the-box. It could help to decide which tool / interpreter / language to use:

  1. Zeppelin native?
  2. language native (e.g. R core graphics plot etc.)?
  3. dedicated libraries (Python: matplotlib [w or w/o %matplotlib inline like jupyter?]; bokeh; R: ggplot2 etc.)
  4. interactive visualization (R: htmlwidgets; relevant visualisations in Bokeh)

@echarles
Copy link
Copy Markdown
Member Author

echarles commented Feb 8, 2016

2 failures on last build:
spark-1.3: ZeppelinIT.testSparkInterpreterDependencyLoading:229->AbstractZeppelinIT.waitForParagraph:66->AbstractZeppelinIT.pollingWait:92 » Timeout

spark-1.5 ZeppelinIT.testAngularDisplay:151 » NoSuchElement Unable to locate element: {"...

The good news is that the SparkR related build and test are ok for all profiles.

@echarles
Copy link
Copy Markdown
Member Author

echarles commented Feb 8, 2016

@echarles @justsomeone1001 This PR supports ggplot2 and displaying R dataframe in native zeppelin widget

ggplot2

Plot

dataframe in native zeppelin widget

Interactive Plot

The best would be to start a thread on the Zeppelin development list to talk about further development and requirement.

@ghost
Copy link
Copy Markdown

ghost commented Feb 8, 2016

@echarles You could simply have answered him "no, this PR doesn't support R visualizations."

I am disinclined to start a thread. I feel like by pointing out the features that your implementation is missing but are present in mine, I'm just helping you compete.

There isn't any benefit to the project from doing that, since what you've built is a subset of what's already in PR 208 and the revision on git.

As you know, there's a moratorium on discussions about PR 208, which is what you based this one on. I, for one, am happy for the break. Perhaps you could consider respecting the moratorium for a few days?

@Leemoonsoo
Copy link
Copy Markdown
Member

Moratorium on discussion about PR 208 is over (a day ago), and it was for only me and Amos.
Zeppelin community never blocks discussion without good reason.

So anyone feel free to comment this PR.

@echarles
Copy link
Copy Markdown
Member Author

Last commit brings the ability to define the ggplot2 image size as asked on https://github.com/datalayer/zeppelin-R/issues/2

I have also added more samples on interactive visualization.

ggplot2

rcharts

maps

googlevis

@echarles
Copy link
Copy Markdown
Member Author

Travis fails with

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.471 sec <<< FAILURE! - in org.apache.zeppelin.spark.SparkRInterpreterTest
testSuccess(org.apache.zeppelin.spark.SparkRInterpreterTest)  Time elapsed: 0.496 sec  <<< ERROR!
java.lang.ExceptionInInitializerError: null
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:186)
    at java.lang.ProcessImpl.start(ProcessImpl.java:130)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
    at scala.sys.process.ProcessBuilderImpl$Simple.run(ProcessBuilderImpl.scala:68)
    at org.ddahl.rscala.callback.RClient$.apply(RClient.scala:689)
    at org.ddahl.rscala.callback.RClient$.apply(RClient.scala:672)
    at org.apache.zeppelin.spark.ZeppelinR$.<init>(ZeppelinR.scala:40)
    at org.apache.zeppelin.spark.ZeppelinR$.<clinit>(ZeppelinR.scala)
    at org.apache.zeppelin.spark.ZeppelinR.css(ZeppelinR.scala)
    at org.apache.zeppelin.spark.SparkRInterpreter.format(SparkRInterpreter.java:179)
    at org.apache.zeppelin.spark.SparkRInterpreter.interpret(SparkRInterpreter.java:134)
    at org.apache.zeppelin.spark.SparkRInterpreterTest.testSuccess(SparkRInterpreterTest.java:83)

Strangely, it was able to pass at least one time 2 days ago.
I will double check on a platform that does not have R and mock more if needed.

@jeffsteinmetz
Copy link
Copy Markdown
Contributor

Just tested, works great. %r shows up in the header now.

@jeffsteinmetz
Copy link
Copy Markdown
Contributor

@echarles I updated the R notebook tutorial with a minor improvement in this patch:
datalayer-attic#9

I noticed passing variables between scala and R is working quite well.
I previously thought there was an issue passing a scala Array to R, but it was because the Array was being treated as a multi-dimensional list in R.
I updated the R tutorial notebook with a small addition of unlist to flatten the array to show the variable as a single list.

for example:

%spark
val a: Array[Double] = Array[Double](30.0, 20.0)
z.put("a", a)

%r
aa <- unlist(z.get("a"))
print(aa)

@echarles
Copy link
Copy Markdown
Member Author

@jeffsteinmetz thx, just pushed a change to show the unlist example.

@Leemoonsoo
Copy link
Copy Markdown
Member

Appreciate @echarles for the R interpreter implementation and very open communication.
And thanks @jeffsteinmetz for providing great review and verification.

I have tested and confirming
it works with default configuration (without SPARK_HOME defined),
it works with SPARK_HOME defined,
it works when binary distribution is generated with -Pbuild-distr flag.

Looks good to me.

@jeffsteinmetz
Copy link
Copy Markdown
Contributor

Looks great.
I just fired up a spark cluster on EC2 using the spark Amazon ec2 scripts.
Pointed zeppelin to the cluster master running 3 nodes.

Built the zeppelin distribution with
mvn clean package -Pspark-1.6 -Ppyspark -Phadoop-2.4 -Psparkr -DskipTests -Pbuild-distr

Shipped the distribution .tar.gz to the clusters master, unzipped, then

# change port to 8090 since spark UI uses 8080
# the ec2 script installs spark in /root/spark
touch conf/zeppelin-env.sh
echo "export ZEPPELIN_PORT=8090" >> conf/zeppelin-env.sh
echo "export SPARK_HOME=/root/spark" >> conf/zeppelin-env.sh
bin/zeppelin-daemon.sh start

Worked as expected.

Looks good to me.

@Leemoonsoo
Copy link
Copy Markdown
Member

Merge into master if there're no more discussions

@prabhjyotsingh
Copy link
Copy Markdown
Contributor

@echarles rebase with master.

@echarles
Copy link
Copy Markdown
Member Author

echarles commented Apr 1, 2016

Thx @prabhjyotsingh for the hint. I have merged and there is no more conflict.

Travis failed for 1.1.1 profile (issue while downloading a jar), otherwise the other profiles are green. How can I retrigger the build without pushing nor closing/opening?

@echarles
Copy link
Copy Markdown
Member Author

echarles commented Apr 2, 2016

Back to green after simple build retrigger.

LGTM for merge.

@jeffsteinmetz
Copy link
Copy Markdown
Contributor

Looks like we are trying hit a moving target on master, specifcially with .travis.yml.

Is it possible for a committer to merge this soon after the branch conflicts are resolved (again) so that @echarles can take this off his plate?

asfgit pushed a commit that referenced this pull request Apr 3, 2016
### What is this PR for?
Improvement of Virtual Machine Script to support R interpretor

### What type of PR is it?
Improvement of Virtual Machine Script to support R interpretor

### Todos
* [x] - Test with #208
* [x] - Test with #702

### Is there a relevant Jira issue?
Zeppelin-700

### How should this be tested?
Follow the steps in this Read Me to build a VM from scratch:
https://github.com/apache/incubator-zeppelin/blob/master/scripts/vagrant/zeppelin-dev/README.md

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?  Added to all headers
* Is there breaking changes for older versions?  No
* Does this needs documentation?  Yes, this will be a separate PR to update docs and README

Author: Jeff Steinmetz <[email protected]>

Closes #751 from jeffsteinmetz/ZEPPELIN-700 and squashes the following commits:

e03dba5 [Jeff Steinmetz] update to support R interpreter sparkr build profile
a9a2052 [Jeff Steinmetz] add base64enc, repr and htmltools
bdcdf5f [Jeff Steinmetz] removed packages not required for pr702.  repr not needed - base64encode not needed - htmltools not needed
d497994 [Jeff Steinmetz] fix so that devtools will install
5643fc6 [Jeff Steinmetz] plotting in r interpreter requires the repr package.  install devtools package first so repr can be installed.
fb18a2b [Jeff Steinmetz] plotting in r interpreter requires the repr package.  install devtools package first so repr can be installed.
ef8f638 [Jeff Steinmetz] ZEPPELIN-700.  Add Ansible R role to Virtual Machine to support dependencies for the R Interpreter
b940bcd [Jeff Steinmetz] ZEPPELIN-700.  Add Ansible R role to Virtual Machine to support dependencies for the R Interpreter
lresende pushed a commit to lresende/incubator-zeppelin that referenced this pull request Apr 3, 2016
### What is this PR for?
Improvement of Virtual Machine Script to support R interpretor

### What type of PR is it?
Improvement of Virtual Machine Script to support R interpretor

### Todos
* [x] - Test with apache#208
* [x] - Test with apache#702

### Is there a relevant Jira issue?
Zeppelin-700

### How should this be tested?
Follow the steps in this Read Me to build a VM from scratch:
https://github.com/apache/incubator-zeppelin/blob/master/scripts/vagrant/zeppelin-dev/README.md

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?  Added to all headers
* Is there breaking changes for older versions?  No
* Does this needs documentation?  Yes, this will be a separate PR to update docs and README

Author: Jeff Steinmetz <[email protected]>

Closes apache#751 from jeffsteinmetz/ZEPPELIN-700 and squashes the following commits:

e03dba5 [Jeff Steinmetz] update to support R interpreter sparkr build profile
a9a2052 [Jeff Steinmetz] add base64enc, repr and htmltools
bdcdf5f [Jeff Steinmetz] removed packages not required for pr702.  repr not needed - base64encode not needed - htmltools not needed
d497994 [Jeff Steinmetz] fix so that devtools will install
5643fc6 [Jeff Steinmetz] plotting in r interpreter requires the repr package.  install devtools package first so repr can be installed.
fb18a2b [Jeff Steinmetz] plotting in r interpreter requires the repr package.  install devtools package first so repr can be installed.
ef8f638 [Jeff Steinmetz] ZEPPELIN-700.  Add Ansible R role to Virtual Machine to support dependencies for the R Interpreter
b940bcd [Jeff Steinmetz] ZEPPELIN-700.  Add Ansible R role to Virtual Machine to support dependencies for the R Interpreter
@echarles
Copy link
Copy Markdown
Member Author

echarles commented Apr 4, 2016

Back to green.

Good to merge.

@jeffsteinmetz
Copy link
Copy Markdown
Contributor

@asfgit or @bzz How do you feel about merging this?
I see it's been 5 days since lazy consensus.
The recent changes have been to keep up with Master merge conflicts.

@bzz
Copy link
Copy Markdown
Member

bzz commented Apr 5, 2016

@jeffsteinmetz thank you for a kind notice! I think it has been reviewed by PPMC already but I'll be glad to make a final pass today and post back the results.

@bzz
Copy link
Copy Markdown
Member

bzz commented Apr 5, 2016

Looks great to me. Let's merge it in.

@asfgit asfgit closed this in 7d6cc7e Apr 5, 2016
asfgit pushed a commit that referenced this pull request Apr 5, 2016
### What is this PR for?
Recently #208 and #702 are merged into master branch.
They passed the CI test individually, but failing after both merged.

* zeppelin-web build error
```
[ERROR] npm ERR! registry error parsing json
[ERROR] npm http 200 https://registry.npmjs.org/bower/1.7.2
[ERROR] npm ERR! SyntaxError: Unexpected token �
[ERROR] npm ERR! ������Y[o�6��+��6��u��lE���
[ERROR] npm ERR! �{h��C�ˑ�L�=RJ���o�9b�4����W4��["��wn�����E����2C�ϕn���U`���a�
[ERROR] npm ERR! G�p^��$�e��Ley,��IU�"/K�,qr�[8�F���^���p�������Z�����=x?�}��{W�+���ܳЀ쵱���}
```

* 'SparkRInterpreter.java' and 'RRepl.java' uses the same interpreter name. 'spark.r'.
That conflicts and make https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java#L87 fails.

* R.md and r.md both exists under same directory. That confuses git client.

### What type of PR is it?
Hot Fix

### Todos
* [x] - Merge R.md and r.md
* [x] - Fix zeppelin-web build error
* [x] - Change interpreter listing order

### What is the Jira issue?

### How should this be tested?

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Lee moon soo <[email protected]>

This patch had conflicts when merged, resolved by
Committer: Lee moon soo <[email protected]>

Closes #815 from Leemoonsoo/r_hotfix and squashes the following commits:

eeb411e [Lee moon soo] Change interpreter listing order
9baf57b [Lee moon soo] Change node and npm version
6854ac7 [Lee moon soo] R.md -> r.md
onkarshedge pushed a commit to onkarshedge/incubator-zeppelin that referenced this pull request May 11, 2016
### What is this PR for?
Improvement of Virtual Machine Script to support R interpretor

### What type of PR is it?
Improvement of Virtual Machine Script to support R interpretor

### Todos
* [x] - Test with apache#208
* [x] - Test with apache#702

### Is there a relevant Jira issue?
Zeppelin-700

### How should this be tested?
Follow the steps in this Read Me to build a VM from scratch:
https://github.com/apache/incubator-zeppelin/blob/master/scripts/vagrant/zeppelin-dev/README.md

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?  Added to all headers
* Is there breaking changes for older versions?  No
* Does this needs documentation?  Yes, this will be a separate PR to update docs and README

Author: Jeff Steinmetz <[email protected]>

Closes apache#751 from jeffsteinmetz/ZEPPELIN-700 and squashes the following commits:

e03dba5 [Jeff Steinmetz] update to support R interpreter sparkr build profile
a9a2052 [Jeff Steinmetz] add base64enc, repr and htmltools
bdcdf5f [Jeff Steinmetz] removed packages not required for pr702.  repr not needed - base64encode not needed - htmltools not needed
d497994 [Jeff Steinmetz] fix so that devtools will install
5643fc6 [Jeff Steinmetz] plotting in r interpreter requires the repr package.  install devtools package first so repr can be installed.
fb18a2b [Jeff Steinmetz] plotting in r interpreter requires the repr package.  install devtools package first so repr can be installed.
ef8f638 [Jeff Steinmetz] ZEPPELIN-700.  Add Ansible R role to Virtual Machine to support dependencies for the R Interpreter
b940bcd [Jeff Steinmetz] ZEPPELIN-700.  Add Ansible R role to Virtual Machine to support dependencies for the R Interpreter
onkarshedge pushed a commit to onkarshedge/incubator-zeppelin that referenced this pull request May 11, 2016
### What is this PR for?

Implement R and SpakR Intepreter as part of the Spark Interpreter Group. It also implements R and Scala binding (in both directions).

### What type of PR is it?

[Feature]

### Todos

* [ ] - Documentation
* [ ] - Unit test (if relevant, as we depend on R being available on the host)
* [ ] - Assess licensing (a priori ok as we don't delive anything out of ASL2, but we should corectly phrase the NOTICE as we depend on non-ASL2 comptabile licenses (R) to run the interpreter).

### Is there a relevant Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-156 SparkR support

### How should this be tested?

You need R available on the host running the notebook.

    For Centos: yum install R R-devel
    For Ubuntu: apt-get install r-base r-cran-rserve

Install additional R packages:

```
curl https://cran.r-project.org/src/contrib/Archive/rscala/rscala_1.0.6.tar.gz -o /tmp/rscala_1.0.6.tar.gz
R CMD INSTALL /tmp/rscala_1.0.6.tar.gz
R -e "install.packages('ggplot2', repos = 'http://cran.us.r-project.org')"
R -e install.packages('knitr', repos = 'http://cran.us.r-project.org')
```

- Build + launch Zeppelin and test the R note.

!!! you need rscala_1.0.6 (if not, you need to build with -Drscala.version=...)

### Screenshots (if appropriate)

#### Simple R

[![Simple R](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/simple-r.png)](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/simple-r.png)

#### Plot

[![Plot](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/plot.png)](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/plot.png)

#### Scala R Binding

[![Scala R Binding](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/scala-r.png)](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/scala-r.png)

#### R Scala Binding

[![R Scala Binding](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/r-scala.png)](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/r-scala.png)

#### SparkR

[![SparkR](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/sparkr.png)](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/sparkr.png)

### Questions:

* Does the licenses files need update? to be checked... (cfr R needs to be available to make this interpreter operational).
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes

Author: Eric Charles <[email protected]>
Author: Lee moon soo <[email protected]>

This patch had conflicts when merged, resolved by
Committer: Lee moon soo <[email protected]>

Closes apache#702 from echarles/rscala-z and squashes the following commits:

137040c [Eric Charles] Trigger build again
53645f6 [Eric Charles] Trigger build
519f3a9 [Eric Charles] Add back the spark.bin.download.url property for sparkr profile
d1f0521 [Eric Charles] Merge with master
72ab72c [Eric Charles] Trigger travis build
151af0a [Eric Charles] Enable back the R install in travis
ed70820 [Eric Charles] Merge with master
ae21036 [Eric Charles] Note: unlist array passed from scala to R, patch contributed by @jeffsteinmetz
ac0e16d [Eric Charles] Merge pull request apache#8 from Leemoonsoo/rscala-z-fix-interpreter-list-order
90c1b4d [Lee moon soo] Sort interpreter list correctly
c36fe8a [Eric Charles] Return SUCCESS if result is empty
463c066 [Eric Charles] Log exception on open
f6e685a [Eric Charles] Remove SparkRInterpeterTest, test is convered in ZeppelinSparkClusterTest
19ec4f3 [Eric Charles] Add back SparkRInterpreterTest.java
58227e9 [Eric Charles] Merge branch 'rscala-z-rs' into rscala-z
290289f [Eric Charles] Merge with master
eb6c40c [Eric Charles] Merge remote rscala-z-rs
962d0d9 [Eric Charles] DOC: Add more visualization libraries
9b95d60 [Lee moon soo] Remove unnecessary test
894c399 [Lee moon soo] Remove SparkRInterpreterTest
bac1e1b [Lee moon soo] R on travis
f6661c2 [Lee moon soo] fix profile activation
1a0195f [Lee moon soo] exclude test when sparkr profile is not defined
2307115 [Lee moon soo] Make sparkr work without SPARK_HOME
dcfee32 [Lee moon soo] Add test
aa35a83 [Lee moon soo] Download sparkR package
1d99fa8 [Lee moon soo] Remove rscala related stuff from project
fc66da9 [Lee moon soo] R -> r
a122894 [Lee moon soo] render output
9df9535 [Lee moon soo] Remove rscala dependency
1e2c99b [Eric Charles] DOC: no need for r-cran-rserve, thx to @jeffsteinmetz
2300ebc [Eric Charles] Update to latest change in interpeter constructs
ecf8bc4 [Eric Charles] Merge with master
a119b72 [Eric Charles] Merge with remote
454c1cb [Eric Charles] Rebase on master and update test to deal with interpretercontext constructor change
b30f6f4 [Eric Charles] Support HTML, TABLE and IMG display - Dynamic form in progress
89d6a3a [Eric Charles] DOC: Fix typo (contributed by @AhyoungRyu) + fix missing depencies (contributed by @btiernay on https://github.com/datalayer/datalayer-zeppelin/issues/6)
a6a3695 [Eric Charles] fix SparkInterpreterTest to deal with previous commit chanching the returned html
35486c7 [Eric Charles] Always return html preview in case of pure text
a0306fc [Eric Charles] Fix the expected html value for test
47eec88 [Eric Charles] Fix code format to make checkstyle happy
f963e1c [Eric Charles] polish examples with title
6cf8615 [Eric Charles] Make it work also on chromium
7b04b6b [Eric Charles] Support ggplot2 output size https://github.com/datalayer/zeppelin-R/issues/2
17d6b0d [Eric Charles] Less test in the SparkRInterpreterTest
f4aac04 [Eric Charles] Add interactive visualization example
816f4d9 [Eric Charles] Run only one test method and see Travis reaction
702556f [Eric Charles] RAT: Disabel RAT check on downloaded rscala folder
d5538a2 [Eric Charles] update pom to download rscala on build
40efe33 [Eric Charles] Add test for SparkRInterpreter
09bb458 [Eric Charles] Use java factory to allow mockito usage
5385adb [Eric Charles] Add jar in R folder
c0063fc [Eric Charles] Ignore downloaded rscala
4d5cfa5 [Eric Charles] Initial documentation for R Interpreter
3e24d02 [Eric Charles] Add license for rscala
c88a914 [Eric Charles] Remove rscala jar
554bcb6 [Eric Charles] Add README.md placeholder for rscala lib folder
40b4ec6 [Eric Charles] Remove png files and restore README.md
220fe51 [Eric Charles] Make rscala configurable in the spark-dependencies module
15375eb [Eric Charles] Add SparRInterpreter implementation
4161619 [Eric Charles] Support HTML, TABLE and IMG display - Dynamic form in progress
8e635e1 [Eric Charles] DOC: Fix typo (contributed by @AhyoungRyu) + fix missing depencies (contributed by @btiernay on https://github.com/datalayer/datalayer-zeppelin/issues/6)
9a988f9 [Eric Charles] fix SparkInterpreterTest to deal with previous commit chanching the returned html
28fc9b2 [Eric Charles] Always return html preview in case of pure text
e8ed8dd [Eric Charles] Fix the expected html value for test
facc682 [Eric Charles] Fix code format to make checkstyle happy
9b168ff [Eric Charles] polish examples with title
8b059c4 [Eric Charles] Make it work also on chromium
66c3545 [Eric Charles] Support ggplot2 output size https://github.com/datalayer/zeppelin-R/issues/2
3ae0bc1 [Eric Charles] Less test in the SparkRInterpreterTest
b9b2787 [Eric Charles] Add interactive visualization example
9218d65 [Eric Charles] Run only one test method and see Travis reaction
ee6e43b [Eric Charles] RAT: Disabel RAT check on downloaded rscala folder
8d664f6 [Eric Charles] update pom to download rscala on build
21668b3 [Eric Charles] Add test for SparkRInterpreter
068ac24 [Eric Charles] Use java factory to allow mockito usage
caf157b [Eric Charles] Add jar in R folder
1eddadb [Eric Charles] Ignore downloaded rscala
0af2bec [Eric Charles] Initial documentation for R Interpreter
8e3c997 [Eric Charles] Add license for rscala
7a95ef4 [Eric Charles] Remove rscala jar
b8ae4eb [Eric Charles] Add README.md placeholder for rscala lib folder
aa6a7a1 [Eric Charles] Remove png files and restore README.md
9312a0c [Eric Charles] Make rscala configurable in the spark-dependencies module
363b244 [Eric Charles] Add SparRInterpreter implementation
onkarshedge pushed a commit to onkarshedge/incubator-zeppelin that referenced this pull request May 11, 2016
### What is this PR for?
Recently apache#208 and apache#702 are merged into master branch.
They passed the CI test individually, but failing after both merged.

* zeppelin-web build error
```
[ERROR] npm ERR! registry error parsing json
[ERROR] npm http 200 https://registry.npmjs.org/bower/1.7.2
[ERROR] npm ERR! SyntaxError: Unexpected token �
[ERROR] npm ERR! ������Y[o�6��+��6��u��lE���
[ERROR] npm ERR! �{h��C�ˑ�L�=RJ���o�9b�4����W4��["��wn�����E����2C�ϕn���U`���a�
[ERROR] npm ERR! G�p^��$�e��Ley,��IU�"/K�,qr�[8�F���^���p�������Z�����=x?�}��{W�+���ܳЀ쵱���}
```

* 'SparkRInterpreter.java' and 'RRepl.java' uses the same interpreter name. 'spark.r'.
That conflicts and make https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java#L87 fails.

* R.md and r.md both exists under same directory. That confuses git client.

### What type of PR is it?
Hot Fix

### Todos
* [x] - Merge R.md and r.md
* [x] - Fix zeppelin-web build error
* [x] - Change interpreter listing order

### What is the Jira issue?

### How should this be tested?

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Lee moon soo <[email protected]>

This patch had conflicts when merged, resolved by
Committer: Lee moon soo <[email protected]>

Closes apache#815 from Leemoonsoo/r_hotfix and squashes the following commits:

eeb411e [Lee moon soo] Change interpreter listing order
9baf57b [Lee moon soo] Change node and npm version
6854ac7 [Lee moon soo] R.md -> r.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.