Apache Zeppelin with Apache Spark clusters.

Overview

Apache Zeppelin is quite working well with Apache Spark and
there are three types of cluster mode in Apache Spark.

This page is for building Apache Spark cluster environments easily using Docker.

We assume you have already installed Zeppelin and docker on your environment. If not, please see here first.

*. This is not the page for optimizing Apache Spark.

Zeppelin with Spark standalone

Requirements

• Apache Zeppelin and Apache Spark use same port of 8080 so you can change the Apache Zeppelin default port you want in conf/zeppelin-site.xml.

Step 1: Download the Spark Standalone Dockerfile here.

]# git clone https://github.com/astroshim/zeppelin_backends_docker.git

Step 2: Build Dockerfile of Spark Standalone.

]# docker build -t "spark_standalone" .

Step 3: Run Docker of Spark Standalone.

]# docker run -it \
-p 8080:8080 \
-p 7077:7077 \
-p 8888:8888 \
-p 8081:8081 \
-h sparkmaster \
--name spark_standalone \
spark_standalone bash; 

Step 4: Configuration of Zeppelin.

• Set master as spark://localhost:7077 on your interpreter configuration page.

Step 5: Run zeppelin with Spark Standalone.

• Run paragraph.

You can see the Running Application on http://localhost:8080/.