[ZEPPELIN-1279] Zeppelin with CDH5.x docker document. by astroshim · Pull Request #1451 · apache/zeppelin

astroshim · 2016-09-22T13:16:32Z

What is this PR for?

This PR is for the documentation of running zeppelin with CDH docker environment.
and This PR is the part of https://issues.apache.org/jira/browse/ZEPPELIN-1198.

Tested CDH5.7 on ubuntu.

What type of PR is it?

Documentation

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-1281

Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

felixcheung · 2016-09-23T02:57:10Z

docs/install/spark_cluster_mode.md


+
+## Zeppelin on CDH
+Cloudera officially provide docker container so we can easily build CDH docker environment following the [link](https://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html).


link to https://www.cloudera.com/documentation/enterprise/latest.html
the latest CDH release is 5.8

Thank you for properly pointing out.

felixcheung · 2016-09-23T02:58:52Z

docs/install/spark_cluster_mode.md

+<img src="../assets/themes/zeppelin/img/docs-img/zeppelin_yarn_conf.png" />
+
+### 4. Run Zeppelin with Spark interpreter
+After running a single paragraph with Spark interpreter in Zeppelin, browse `http://<hostname>:8088/cluster/apps` and check Zeppelin application is running well or not.


should this explain to check zeppelin (http://:8080) also?

Okay. Let me add that. Thanks.

AhyoungRyu · 2016-09-23T04:43:14Z

@astroshim Thanks for your contribution! This can be really helpful to CDH & Spark & Zeppelin users.

How about separating Zeppelin on Spark Cluster Mode page with Zeppelin on CDH instead of putting this contents in one page? Then you need to create docs/install/cdh.md maybe.
I think it's more makes to have only Spark Cluster Mode contents in docs/install/spark_cluster_mode.md.

Or there is an another option. If you think the contents in Zeppelin on CDH is too short to have a separated page, you can just modify the title("Apache Zeppelin on Spark Cluster Mode") and the contents of "Overview" in spark_cluster_mode.md. And maybe the file name needs to be changed accordingly :)

What do you think?

astroshim · 2016-09-26T05:18:42Z

@AhyoungRyu Thank you for your suggestion. I think creating new page for Zeppelin on CDH is better, what do you think?

AhyoungRyu · 2016-09-26T05:25:13Z

@astroshim Yeah that would be better than all contents are together in one page actually. Thanks for your consideration :D

Minor update for cdh.md

AhyoungRyu · 2016-09-27T01:42:31Z

docs/install/cdh.md

+export SPARK_HOME=[your_spark_home_path]
+```
+
+`HADOOP_CONF_DIR`(Hadoop configuration path) is defined in `/scripts/docker/spark-cluster-managers/cdh/hdfs_conf`.


@astroshim Do you mean /scripts/docker/spark-cluster-managers/cdh/*.xml?

The configuration for cdh is in the ``/scripts/docker/spark-cluster-managers/cdh/hdfs_conf/` like yarn mode.

Sorry, I made /scripts/docker/spark-cluster-managers/cdh/hdfs_conf/ and move configurations to that path.

@astroshim Thanks for your prompt handling :D

AhyoungRyu · 2016-09-27T01:58:29Z

@astroshim To make CI happy, you might need to exclude scripts dir from the rat-plugin. Please add this single line<exclude>scripts/**</exclude> to here.

AhyoungRyu · 2016-09-27T05:58:16Z

@astroshim still CI is not happy but it seems irrelevant to this change. As below result says, there is a test failure in zeppelin-display.

36mTests: succeeded 17, failed 1, canceled 0, ignored 0, pending 0�[0m
�[31m*** 1 TEST FAILED ***�[0m
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Zeppelin ........................................... SUCCESS [  4.051 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [02:13 min]
[INFO] Zeppelin: Zengine .................................. SUCCESS [01:53 min]
[INFO] Zeppelin: Display system apis ...................... FAILURE [ 16.048 s]
[INFO] Zeppelin: Spark dependencies ....................... SKIPPED
[INFO] Zeppelin: Spark .................................... SKIPPED
[INFO] Zeppelin: Markdown interpreter ..................... SKIPPED
[INFO] Zeppelin: Angular interpreter ...................... SKIPPED
[INFO] Zeppelin: Shell interpreter ........................ SKIPPED
[INFO] Zeppelin: Livy interpreter ......................... SKIPPED
[INFO] Zeppelin: HBase interpreter ........................ SKIPPED
[INFO] Zeppelin: PostgreSQL interpreter ................... SKIPPED
[INFO] Zeppelin: JDBC interpreter ......................... SKIPPED
[INFO] Zeppelin: File System Interpreters ................. SKIPPED
[INFO] Zeppelin: Flink .................................... SKIPPED
[INFO] Zeppelin: Apache Ignite interpreter ................ SKIPPED
[INFO] Zeppelin: Kylin interpreter ........................ SKIPPED
[INFO] Zeppelin: Python interpreter ....................... SKIPPED
[INFO] Zeppelin: Lens interpreter ......................... SKIPPED
[INFO] Zeppelin: Apache Cassandra interpreter ............. SKIPPED
[INFO] Zeppelin: Elasticsearch interpreter ................ SKIPPED
[INFO] Zeppelin: BigQuery interpreter ..................... SKIPPED
[INFO] Zeppelin: Alluxio interpreter ...................... SKIPPED
[INFO] Zeppelin: web Application .......................... SKIPPED
[INFO] Zeppelin: Server ................................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO] Zeppelin: R Interpreter ............................ SKIPPED
[INFO] Zeppelin: Scalding interpreter ..................... SKIPPED
[INFO] Zeppelin: Examples ................................. SKIPPED
[INFO] Zeppelin: Example application - Clock .............. SKIPPED
[INFO] Zeppelin: Example application - Horizontal Bar chart SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:28 min
[INFO] Finished at: 2016-09-27T02:59:19+00:00
[INFO] Final Memory: 33M/421M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project zeppelin-display_2.11: There are test failures -> [Help 1]
[ERROR]

Looks good 👍

astroshim · 2016-09-28T14:12:37Z

@AhyoungRyu Thank you so much for taking care of this!!

AhyoungRyu · 2016-09-29T02:24:41Z

@astroshim Appreciate for your effort! Merge into master if there are no more discussions :)

### What is this PR for? This PR is for the documentation of running zeppelin with CDH docker environment. and This PR is the part of https://issues.apache.org/jira/browse/ZEPPELIN-1198. Tested CDH5.7 on ubuntu. ### What type of PR is it? Documentation ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1281 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <[email protected]> Author: AhyoungRyu <[email protected]> Author: HyungSung <[email protected]> Closes apache#1451 from astroshim/ZEPPELIN-1281 and squashes the following commits: 5dcb8c1 [astroshim] move configurations to right path and add excluding rat-plugin 09408e3 [HyungSung] Merge pull request apache#11 from AhyoungRyu/ZEPPELIN-1281-ahyoung 850119c [AhyoungRyu] Generate TOC & change some sentences e687a53 [AhyoungRyu] Replace zeppelin_with_cdh.png to crop the url part cc9a023 [AhyoungRyu] Remove main title link anchor b525f68 [astroshim] separate cdh doc with spark_cluster_mode.md e66993f [astroshim] fix doc a7b5b2d [astroshim] cdh docker environment

astroshim added 2 commits September 22, 2016 16:55

cdh docker environment

a7b5b2d

fix doc

e66993f

felixcheung reviewed Sep 23, 2016

View reviewed changes

astroshim and others added 5 commits September 26, 2016 15:51

separate cdh doc with spark_cluster_mode.md

b525f68

Remove main title link anchor

cc9a023

Replace zeppelin_with_cdh.png to crop the url part

e687a53

Generate TOC & change some sentences

850119c

Merge pull request #11 from AhyoungRyu/ZEPPELIN-1281-ahyoung

09408e3

Minor update for cdh.md

AhyoungRyu reviewed Sep 27, 2016

View reviewed changes

move configurations to right path and add excluding rat-plugin

5dcb8c1

asfgit closed this in c7ce709 Sep 29, 2016



		## Zeppelin on CDH
		Cloudera officially provide docker container so we can easily build CDH docker environment following the [link](https://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html).

Conversation

astroshim commented Sep 22, 2016

What is this PR for?

What type of PR is it?

What is the Jira issue?

Questions:

Uh oh!

felixcheung Sep 23, 2016

Choose a reason for hiding this comment

Uh oh!

astroshim Sep 26, 2016

Choose a reason for hiding this comment

Uh oh!

felixcheung Sep 23, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

astroshim Sep 26, 2016

Choose a reason for hiding this comment

Uh oh!

AhyoungRyu commented Sep 23, 2016

Uh oh!

astroshim commented Sep 26, 2016

Uh oh!

AhyoungRyu commented Sep 26, 2016

Uh oh!

AhyoungRyu Sep 27, 2016

Choose a reason for hiding this comment

Uh oh!

astroshim Sep 27, 2016

Choose a reason for hiding this comment

Uh oh!

astroshim Sep 27, 2016

Choose a reason for hiding this comment

Uh oh!

AhyoungRyu Sep 27, 2016

Choose a reason for hiding this comment

Uh oh!

AhyoungRyu commented Sep 27, 2016

Uh oh!

AhyoungRyu commented Sep 27, 2016

Uh oh!

astroshim commented Sep 28, 2016

Uh oh!

AhyoungRyu commented Sep 29, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

felixcheung Sep 23, 2016 •

edited

Loading