Skip to content

Add force layout visualization in paragraphs for SQL queries#1135

Closed
davidtzoor wants to merge 8 commits intoapache:masterfrom
davidtzoor:forcelayout
Closed

Add force layout visualization in paragraphs for SQL queries#1135
davidtzoor wants to merge 8 commits intoapache:masterfrom
davidtzoor:forcelayout

Conversation

@davidtzoor
Copy link
Copy Markdown

@davidtzoor davidtzoor commented Jul 5, 2016 •
edited
Loading

What is this PR for?

Add force layout (graph) visualization to the existing set of visualizations in Zeppelin Web notp.

What type of PR is it?

[Feature]

PR description

data model used by the graph:

{
    nodes: [
        { name, group },
        ...
    ],
    links: [
        { source, target }
    ]
}

node.name is the unique identifier of a node, the group determines the color of the node.
link.source/target are the start and end points of the edges of the graph, based on the nodes names.

What is the Jira issue?

[ZEPPELIN-1011]

How should this be tested?

Run the following 2 code samples in a paragraph to generate the data set in the context:


data = """7901850786671431223   A   -9201925366945671176        B
-8214570892815727335    A   -9142458413787123325        B
9209298202739151797 A   -9142458413787123325        B
-235707035423510460 A   -9142458413787123325        B
-4432871508352121742    A   -8985197548318453090        B
-5282254351261019420    A   -8951838399863319184        B
-4766152790540222112    A   -8933665362858120936        B
-1111863962775664382    A   -8867555589789477095        B
5476637451143911605 A   -8864983663710882628        B
4361582717736404885 A   -8825261852220531319        B
-7605907640540369084    A   -8825261852220531319        B
-5837573774451340812    A   -8825261852220531319        B
7163053636126215249 A   -8825261852220531319        B
861139451162471159  A   -8825261852220531319        B
4710296859646728237 A   -8825261852220531319        B
-599615685466611445 A   -8799537192434911718        B
-8744540049061656557    A   -8780425837578325961        B
-7395255090693731764    A   -8780425837578325961        B
-8450625258438666747    A   -8776915776245574573        B
-5109485004342136017    A   -8768104402834035581        B
-6412466919921704273    A   -8719754505429102068        B
-5110717923604683987    A   -8719754505429102068        B
-3658773984531314435    A   -8719754505429102068        B
-8450625258438666747    A   -8719501119501924815        B
-6088842233080689224    A   -8719501119501924815        B
5080785082159503629 A   -8719501119501924815        B
-235707035423510460 A   -8719501119501924815        B
4710296859646728237 A   -8719501119501924815        B
574465287884870837  A   -8618660447454744071        B
2627058191174051503 A   -8618660447454744071        B
3361213877988256304 A   -8556158745209629329        B
1753480874442477887 A   -8542160690522263351        B
4710082952441870044 A   -8523724932066627876        B
3593037200969895583 A   -8523724932066627876        B
6549790973119727915 A   -8523724932066627876        B
-6391647869209764907    A   -8512098745672685655        B
6628570063743632012 A   -8511148708034410263        B
5476637451143911605 A   -8346285257963178932        B
4817374285510464702 A   -8301495021220733733        B
-443915960696260770 A   -8301495021220733733        B
8256503083397256159 A   -8297987713158283102        B
-6523542708797056082    A   -8294378768128938939        B
-8450625258438666747    A   -8221781283317332638        B
-5763079946737833038    A   -8221781283317332638        B
8437764668109048693 A   -8217668597515102322        B
-7080088993011396672    A   -8217668597515102322        B
5332146435231274697 A   -8217668597515102322        B
8424739580045126352 A   -8217668597515102322        B
-7605907640540369084    A   -8217668597515102322        B
-4004532099129726555    A   -8217668597515102322        B
4586059026815355905 A   -8217668597515102322        B
-3549137688772712918    A   -8217668597515102322        B
3775141695186132785 A   -8217668597515102322        B
1081087240374379258 A   -8217668597515102322        B
-2484166165546302047    A   -8204410554003120406        B
7196795530660323225 A   -8204410554003120406        B
8945507541332375803 A   -8161835148739852333        B
4113673676745842975 A   -8160815874720256810        B
809496795310563074  A   -8160815874720256810        B
7821859219458408878 A   -8160815874720256810        B
-8333264540320731861    A   -8160815874720256810        B
7088437698409708484 A   -8160815874720256810        B
-235707035423510460 A   -8160815874720256810        B
-4773953531103614805    A   -8160815874720256810        B
9192952707487082353 A   -8160815874720256810        B
207587472337949580  A   -8081710953330287587        B
207587472337949580  A   -8048937451483613113        B
5332146435231274697 A   -8035077502230232796        B
6218300533454972222 A   -8023106060481013958        B
-7607387850802540117    A   -8006424366939731598        B
6628570063743632012 A   -7981526580741804911        B
5549018648153862860 A   -7937163015311011155        B
3537420035516183306 A   -7928986520757371761        B
5171836565124133221 A   -7928986520757371761        B
6193634154424484163 A   -7925817622173741244        B
3361213877988256304 A   -7925817622173741244        B
-235707035423510460 A   -7898905965985334148        B
1901177400995208605 A   -7883519319216828057        B
-8563242677372011407    A   -7841902181988625360        B
1785460983681264737 A   -7839581486700963709        B
-2932784130250822001    A   -7825120470248334761        B
7735790126417584160 A   -7814285974898155285        B
-199144186590703694 A   -7814285974898155285        B
5968949691686201056 A   -7783329245620004076        B
6125353611611344475 A   -7783329245620004076        B
5518918926728439070 A   -7783329245620004076        B
-6391647869209764907    A   -7783329245620004076        B
5080785082159503629 A   -7758062101959114640        B
-2798991516883042219    A   -7758062101959114640        B
809496795310563074  A   -7758062101959114640        B
534292520350604233  A   -7758062101959114640        B
6910051452477481802 A   -7688774361473897237        B
5295618851878447136 A   -7663697366780658174        B
2660413613931574480 A   -7663697366780658174        B
-8450625258438666747    A   -7649960976820763242        B
4817374285510464702 A   -7649960976820763242        B
6621281190285958998 A   -7605823123068693377        B
-7358461975222165537    A   -7503340627119474561        B
7735790126417584160 A   -7503340627119474561        B
4738737749005310255 A   -7503340627119474561        B"""

%pyspark 

data2 = [[y  for y in x.split()] for x in data.split("\n")]
distData = sc.parallelize(data2)
#print data2

df = distData.toDF(["src","type1","dst","type2"])

df.registerTempTable("example")

now run the following to get the data set:
%sql select * from example

and finally choose the source and destination columns to generate the graph:
image

Screenshot

eeffbb58-3238-11e6-92ea-148a4063d353

Questions:

  • Does the licenses files need update?
    No.
  • Is there breaking changes for older versions?
    No.
  • Does this needs documentation?
    We can add some documentation explaining about the different fields that need to be selected in order to generate the force layout component.

@davidtzoor davidtzoor mentioned this pull request Jul 5, 2016
Closed
@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Jul 7, 2016

@AhyoungRyu I did exactly as you told me, and all of the other commits were added to my PR. I undid the push and went back to the original commit of the pull request. Can you please advise? thanks

@AhyoungRyu
Copy link
Copy Markdown
Contributor

AhyoungRyu commented Jul 11, 2016

@davidtzoor Oh sorry for my late response. Did you fetch the latest version of upstream master?

git checkout master
git fetch upstream
git pull upstream master

Then check git log in your local and compare with this page.
If those two have same history, you need to rebase with this status

git checkout forcelayout
git rebase master

then try to fix the conflict.

@corneadoug
Copy link
Copy Markdown
Contributor

corneadoug commented Jul 11, 2016

You might also need git push -f after resolving conflicts

@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Jul 11, 2016

@corneadoug are the build errors due to my code?

@corneadoug
Copy link
Copy Markdown
Contributor

corneadoug commented Jul 11, 2016

@davidtzoor No the CI is fine

@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Jul 12, 2016

@corneadoug so is my PR ok?

@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Jul 20, 2016

hi @corneadoug any update on this PR? I am sorry to poke you it. Just wanted to know if there something else I need to do. Thanks 👍

@corneadoug
Copy link
Copy Markdown
Contributor

corneadoug commented Aug 17, 2016

@davidtzoor sorry for the long wait, I was trying to finish a few things before looking closer into this PR.

There was another visualization that got merged, so you will have to rebase.

Can you also provide a few things to make it easier for me to review:

  • Some code (paragraph) that create or import a data set necessary to use that graph, you can update the How should this be tested part of the PR's description with it.
  • The data structure used by the graph (in PR description)

Then I should be able to test it out, and give some feedback to have it merged

@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Aug 17, 2016

@corneadoug great! I will get it done by end of day and update you.

@corneadoug
Copy link
Copy Markdown
Contributor

corneadoug commented Aug 19, 2016

@davidtzoor Take your time, just ping me when its done

@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Aug 21, 2016

@corneadoug I have rebased, added the test data, instructions and data model above. Please let me know what you think

@davidtzoor
Copy link
Copy Markdown
Author

davidtzoor commented Sep 26, 2016

@corneadoug any update on my PR? looking forward to your feedback

@asfgit asfgit closed this in c38a0a0 May 9, 2018
asfgit pushed a commit that referenced this pull request May 9, 2018
@jongyoul
Close outdated PRs
b11d355 
close #83
close #86
close #125
close #133
close #139
close #146
close #193
close #203
close #246
close #262
close #264
close #273
close #291
close #299
close #320
close #347
close #389
close #413
close #423
close #543
close #560
close #658
close #670
close #728
close #765
close #777
close #782
close #783
close #812
close #822
close #841
close #843
close #878
close #884
close #918
close #989
close #1076
close #1135
close #1187
close #1231
close #1304
close #1316
close #1361
close #1385
close #1390
close #1414
close #1422
close #1425
close #1447
close #1458
close #1466
close #1485
close #1492
close #1495
close #1497
close #1536
close #1545
close #1561
close #1577
close #1600
close #1603
close #1678
close #1695
close #1739
close #1748
close #1765
close #1767
close #1776
close #1783
close #1799
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@davidtzoor @AhyoungRyu @corneadoug