sudo: false

cache:

directories:

- .spark-dist

addons:

apt:

sources:

- r-packages-precise

packages:

- r-base-dev

- r-cran-evaluate

- r-cran-base64enc

env: SPARK_VER="1.6.1" HADOOP_VER="2.3" PROFILE="-Pspark-1.6 -Pr -Phadoop-2.3 -Ppyspark -Psparkr -Pscalding" BUILD_FLAG="package -Pbuild-distr" TEST_FLAG="verify -Pusing-packaged-distr" TEST_PROJECTS=""

env: SPARK_VER="1.5.2" HADOOP_VER="2.3" PROFILE="-Pspark-1.5 -Pr -Phadoop-2.3 -Ppyspark -Psparkr" BUILD_FLAG="package -DskipTests" TEST_FLAG="verify" TEST_PROJECTS="-pl zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-display,spark-dependencies,spark,r -Dtest=org.apache.zeppelin.rest.*Test,org.apache.zeppelin.spark* -DfailIfNoTests=false"

env: SPARK_VER="1.4.1" HADOOP_VER="2.3" PROFILE="-Pspark-1.4 -Pr -Phadoop-2.3 -Ppyspark -Psparkr" BUILD_FLAG="package -DskipTests" TEST_FLAG="verify" TEST_PROJECTS="-pl zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-display,spark-dependencies,spark,r -Dtest=org.apache.zeppelin.rest.*Test,org.apache.zeppelin.spark* -DfailIfNoTests=false"

# Test selenium with spark module for 1.6.1

env: TEST_SELENIUM="true" SPARK_VER="1.6.1" HADOOP_VER="2.3" PROFILE="-Pspark-1.6 -Phadoop-2.3 -Ppyspark" BUILD_FLAG="package -DskipTests" TEST_FLAG="verify" TEST_PROJECTS="-pl zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-display,spark-dependencies,spark -Dtest=org.apache.zeppelin.AbstractFunctionalSuite -DfailIfNoTests=false"

- "ls -la .spark-dist"

- mkdir -p ~/R

- echo 'R_LIBS=~/R' > ~/.Renviron

- R -e "install.packages('knitr', repos = 'http://cran.us.r-project.org', lib='~/R')"

- export R_LIBS='~/R'

- tail conf/zeppelin-env.sh

(Apache 2.0) Bootstrap v3.0.2 (http://getbootstrap.com/) - https://github.com/twbs/bootstrap/blob/v3.0.2/LICENSE

========================================================================

BSD 3-Clause licenses

========================================================================

The following components are provided under the BSD 3-Clause license. See file headers and project links for details.

(BSD 3 Clause) portions of rscala 1.0.6 (https://dahl.byu.edu/software/rscala/) - https://cran.r-project.org/web/packages/rscala/index.html

r/R/rzeppelin/R/{common.R, globals.R,protocol.R,rServer.R,scalaInterpreter.R,zzz.R }

r/src/main/scala/org/apache/zeppelin/rinterpreter/rscala/{Package.scala, RClient.scala}

(BSD 3 Clause) portions of Scala (http://www.scala-lang.org/download) - http://www.scala-lang.org/download/#License

r/src/main/scala/scala/Console.scala

ZEPPELIN_CLASSPATH=${SPARK_APP_JAR}

# Need to add the R Interpreter

RZEPPELINPATH="$(ls ${ZEPPELIN_HOME}/interpreter/spark/zeppelin-zr*.jar)"

ZEPPELIN_CLASSPATH="${ZEPPELIN_CLASSPATH}:${RZEPPELINPATH}"

RZEPPELINPATH="$(ls ${ZEPPELIN_HOME}/interpreter/spark/zeppelin-zr*.jar)"

ZEPPELIN_CLASSPATH="${ZEPPELIN_CLASSPATH}:${RZEPPELINPATH}"

elif [[ "${INTERPRETER_ID}" == "hbase" ]]; then

if [[ -n "${HBASE_CONF_DIR}" ]]; then

ZEPPELIN_CLASSPATH+=":${HBASE_CONF_DIR}"

elif [[ -n "${HBASE_HOME}" ]]; then

ZEPPELIN_CLASSPATH+=":${HBASE_HOME}/conf"

else

echo "HBASE_HOME and HBASE_CONF_DIR are not set, configuration might not be loaded"

fi

shiro.loginUrl = /api/login

/api/version = anon

<value>org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkRInterpreter,org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.angular.AngularInterpreter,org.apache.zeppelin.shell.ShellInterpreter,org.apache.zeppelin.hive.HiveInterpreter,org.apache.zeppelin.tajo.TajoInterpreter,org.apache.zeppelin.file.HDFSFileInterpreter,org.apache.zeppelin.flink.FlinkInterpreter,org.apache.zeppelin.lens.LensInterpreter,org.apache.zeppelin.ignite.IgniteInterpreter,org.apache.zeppelin.ignite.IgniteSqlInterpreter,org.apache.zeppelin.cassandra.CassandraInterpreter,org.apache.zeppelin.geode.GeodeOqlInterpreter,org.apache.zeppelin.postgresql.PostgreSqlInterpreter,org.apache.zeppelin.jdbc.JDBCInterpreter,org.apache.zeppelin.phoenix.PhoenixInterpreter,org.apache.zeppelin.kylin.KylinInterpreter,org.apache.zeppelin.elasticsearch.ElasticsearchInterpreter,org.apache.zeppelin.scalding.ScaldingInterpreter,org.apache.zeppelin.alluxio.AlluxioInterpreter,org.apache.zeppelin.hbase.HbaseInterpreter,org.apache.zeppelin.rinterpreter.KnitR,org.apache.zeppelin.rinterpreter.RRepl</value>

<li><a href="{{BASE_PATH}}/interpreter/R.html">R</a></li>

layout: page

title: "R Interpreter"

description: ""

group: manual

{% include JB/setup %}

This is a the Apache (incubating) Zeppelin project, with the addition of support for the R programming language and R-spark integration.

Additional requirements for the R interpreter are:

* R 3.1 or later (earlier versions may work, but have not been tested)

* The `evaluate` R package.

For full R support, you will also need the following R packages:

* `knitr`

* `repr` -- available with `devtools::install_github("IRkernel/repr")`

* `htmltools` -- required for some interactive plotting

* `base64enc` -- required to view R base plots

To run Zeppelin with the R Interpreter, the SPARK_HOME environment variable must be set. The best way to do this is by editing `conf/zeppelin-env.sh`.

If it is not set, the R Interpreter will not be able to interface with Spark.

You should also copy `conf/zeppelin-site.xml.template` to `conf/zeppelin-site.xml`. That will ensure that Zeppelin sees the R Interpreter the first time it starts up.

By default, the R Interpreter appears as two Zeppelin Interpreters, `%r` and `%knitr`.

`%r` will behave like an ordinary REPL. You can execute commands as in the CLI.

[](screenshots/repl2plus2.png)

R base plotting is fully supported

[](screenshots/replhist.png)

If you return a data.frame, Zeppelin will attempt to display it using Zeppelin's built-in visualizations.

[](screenshots/replhead.png)

`%knitr` interfaces directly against `knitr`, with chunk options on the first line:

[](screenshots/knitgeo.png)

[](screenshots/knitstock.png)

[](screenshots/knitmotion.png)

The two interpreters share the same environment. If you define a variable from `%r`, it will be within-scope if you then make a call using `knitr`.

If `SPARK_HOME` is set, the `SparkR` package will be loaded automatically:

[](screenshots/sparkrfaithful.png)

The Spark Context and SQL Context are created and injected into the local environment automatically as `sc` and `sql`.

The same context are shared with the `%spark`, `%sql` and `%pyspark` interpreters:

[](screenshots/backtoscala.png)

You can also make an ordinary R variable accessible in scala and Python:

[](screenshots/varr1.png)

And vice versa:

[](screenshots/varscala.png)

[](screenshots/varr2.png)

* Almost all issues with the R interpreter turned out to be caused by an incorrectly set `SPARK_HOME`. The R interpreter must load a version of the `SparkR` package that matches the running version of Spark, and it does this by searching `SPARK_HOME`. If Zeppelin isn't configured to interface with Spark in `SPARK_HOME`, the R interpreter will not be able to connect to Spark.

* The `knitr` environment is persistent. If you run a chunk from Zeppelin that changes a variable, then run the same chunk again, the variable has already been changed. Use immutable variables.

* (Note that `%spark.r` and `$r` are two different ways of calling the same interpreter, as are `%spark.knitr` and `%knitr`. By default, Zeppelin puts the R interpreters in the `%spark.` Interpreter Group.

* Using the `%r` interpreter, if you return a data.frame, HTML, or an image, it will dominate the result. So if you execute three commands, and one is `hist()`, all you will see is the histogram, not the results of the other commands. This is a Zeppelin limitation.

* If you return a data.frame (for instance, from calling `head()`) from the `%spark.r` interpreter, it will be parsed by Zeppelin's built-in data visualization system.

* Why `knitr` Instead of `rmarkdown`? Why no `htmlwidgets`? In order to support `htmlwidgets`, which has indirect dependencies, `rmarkdown` uses `pandoc`, which requires writing to and reading from disc. This makes it many times slower than `knitr`, which can operate entirely in RAM.

* Why no `ggvis` or `shiny`? Supporting `shiny` would require integrating a reverse-proxy into Zeppelin, which is a task.

* Max OS X & case-insensitive filesystem. If you try to install on a case-insensitive filesystem, which is the Mac OS X default, maven can unintentionally delete the install directory because `r` and `R` become the same subdirectory.

* Error `unable to start device X11` with the repl interpreter. Check your shell login scripts to see if they are adjusting the `DISPLAY` environment variable. This is common on some operating systems as a workaround for ssh issues, but can interfere with R plotting.

* akka Library Version or `TTransport` errors. This can happen if you try to run Zeppelin with a SPARK_HOME that has a version of Spark other than the one specified with `-Pspark-1.x` when Zeppelin was compiled.

[R](https://www.r-project.org) is a free software environment for statistical computing and graphics.

To run R code and visualize plots in Apache Zeppelin, you will need R on your master node (or your dev laptop).

+ For Centos: `yum install R R-devel libcurl-devel openssl-devel`

+ For Ubuntu: `apt-get install r-base`

Validate your installation with a simple R command:

To enjoy plots, install additional libraries with:

We recommend you to also install the following optional R libraries for happy data analytics:

+ glmnet

+ pROC

+ data.table

+ caret

+ sqldf

+ wordcloud