Skip to content

[ZEPPELIN-23] Set version of default spark interpreter build profile from 1.1 to 1.3#1

Merged
ilganeli merged 6 commits intoilganeli:ZEPPELIN-23from
Leemoonsoo:ZEPPELIN-23
Jun 3, 2015
Merged

[ZEPPELIN-23] Set version of default spark interpreter build profile from 1.1 to 1.3#1
ilganeli merged 6 commits intoilganeli:ZEPPELIN-23from
Leemoonsoo:ZEPPELIN-23

Conversation

@Leemoonsoo
Copy link
Copy Markdown

I have activated spark-1.3 profile by default instead of change default spark version from 1.1.1 to 1.3.1. Because of, to make spark 1.3.1 work, not only spark.version but also all related library version should be taken care.

Also unitest are updated to make work with spark 1.3.x

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: To be consistent, we probably want to indent the lines below.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as below, potentially safer to explicitly check the numerical value of the version, e.g.:

String[] split = version.split(".")
int version = Integer.parseInt(split[0]) + Integer.parseInt(split[1])
return version <= 11;

@Leemoonsoo
Copy link
Copy Markdown
Author

@ilganeli Thanks for the feedback! I have updated source code's indentation, also followed your suggestion that check version number in numerical value.

@Leemoonsoo
Copy link
Copy Markdown
Author

Changed to return SparkInterpreterTest.getSparkVersionNumber() >= 13

@ilganeli
Copy link
Copy Markdown
Owner

ilganeli commented Jun 3, 2015

LGTM

ilganeli pushed a commit that referenced this pull request Jun 3, 2015
[ZEPPELIN-23] Set version of default spark interpreter build profile from 1.1 to 1.3
@ilganeli ilganeli merged commit d1d642e into ilganeli:ZEPPELIN-23 Jun 3, 2015
@Leemoonsoo
Copy link
Copy Markdown
Author

Thanks!

ilganeli pushed a commit that referenced this pull request Jul 8, 2015
…from 1.1 to 1.3

I updated the version of spark used by default throughout the dependency listing to be 1.3.0 (to match the version used in the 1.3 profile).

Author: Lee moon soo <[email protected]>
Author: Ilya Ganelin <[email protected]>

This patch had conflicts when merged, resolved by
Committer: Lee moon soo <[email protected]>

Closes apache#71 from ilganeli/ZEPPELIN-23 and squashes the following commits:

d1d642e [Ilya Ganelin] Merge pull request #1 from Leemoonsoo/ZEPPELIN-23
0108781 [Lee moon soo] update isDataFrameSupported()
ac4e8bc [Lee moon soo] check spark version as a numerical number in tests
405da99 [Lee moon soo] Fix indentation
28dd7bf [Lee moon soo] Make unittest pass with spark-1.3 profile
cf2ed0c [Lee moon soo] Activate spark-1.3 profile by default. Set spark 1.3.1 as a default version of spark-1.3 profile
13bf6c1 [Lee moon soo] remove z.load() error for spark 1.2, spark 1.3
1402df2 [Ilya Ganelin] Updated default spark version to 1.3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants