Skip to content

[ZEPPELIN-772] - Improve text append when streaming#804

Closed
corneadoug wants to merge 1 commit intoapache:masterfrom
corneadoug:improve/flushStreamingOutput
Closed

[ZEPPELIN-772] - Improve text append when streaming#804
corneadoug wants to merge 1 commit intoapache:masterfrom
corneadoug:improve/flushStreamingOutput

Conversation

@corneadoug
Copy link
Copy Markdown
Contributor

What is this PR for?

This PR is to improve the result output when having interpreter streaming running.
If you navigate outside the notebook, then come back inside, the streaming output will append below previous result.

What type of PR is it?

Improvement

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-772

How should this be tested?

You can run this

(1 to 40).foreach{ i=>
    Thread.sleep(1000)
    println("Hello " + i)
}

Then navigate to home, then back to the notebook.
You should see the newly streamed result only.

Screenshots

Before:
before

After:
after

Questions:

  • Does the licenses files need update? No
  • Is there breaking changes for older versions? No
  • Does this needs documentation? No

@vgmartinez
Copy link
Copy Markdown
Contributor

I tested....LGTM

@echarles
Copy link
Copy Markdown
Member

echarles commented Apr 2, 2016

The expected behavior I would expect is the "Before" one. If I ask for a result, it should be consistent, whatever page I navigate in between.

I would prefer a discussion on the mailing list to gather user feedback on this.

@felixcheung
Copy link
Copy Markdown
Member

Shouldn't all output be captured? Is there a reason we want to clear it when the user navigates out? Perhaps I'm not fully understanding the behavior change?

@bzz
Copy link
Copy Markdown
Member

bzz commented Apr 4, 2016

Rebasing this on latest master should let current CI failure go.

@corneadoug
Copy link
Copy Markdown
Contributor Author

Maybe the GIFs are not obvious enough :)

The PR is mainly trying to improve the rendering of the streaming capability introduced in #611.

So I will try to explain more in detail what is happening and how things work on the front-end side.

How It Works

First of all, its good to know that this is only a case for TEXT type of return values.

The way the back-end communicates data to the front-end end is through 2 different websocket events:

  • PARAGRAPH_UPDATE_OUTPUT: Which remove previous result and replace it by the event content
  • PARAGRAPH_APPEND_OUTPUT: Which add to the end of previous result

So for example, if we take this code:

(1 to 40).foreach{ i=>
    Thread.sleep(1000)
    println(i) 
}

It would send numbers from 1 to 40 with 1000ms interval, and this is what would happen:

- PARAGRAPH_UPDATE_OUTPUT (sent: "")

  -- PARAGRAPH_APPEND_OUTPUT (sent: "1")

  -- PARAGRAPH_APPEND_OUTPUT (sent: "2")

  -- PARAGRAPH_APPEND_OUTPUT (sent: "3")

  ...

- PARAGRAPH_UPDATE_OUTPUT (sent: "1/n2/n3/n......40")

So the way it works is that the job results are never persisted inside the Notebook before the end of the job.

The Current Problem

Since the temporary results are not persisted, whenever you navigate in and out of the Running Notebook, it will first reload the previous saved result in that paragraph.

So if you did run a loop through the ABC before, The paragraph will render that result.

Then you will receive some PARAGRAPH_APPEND_OUTPUT events from your running job, which will transform the result you see in your paragraph into something like:

A
B
...
Z
21 (PARAGRAPH_APPEND_OUTPUT)
22 (PARAGRAPH_APPEND_OUTPUT)
23 (PARAGRAPH_APPEND_OUTPUT)

Mainly because the flush of previous result (visually) was done in the PARAGRAPH_UPDATE_OUTPUT event at the beginning of the job.

What the PR achieve

In this PR, we try to clear the previous unrelated result of the paragraph when the paragraph receives an PARAGRAPH_APPEND_OUTPUT event for the first time in the Notebook, in order to keep result of that job only.

@corneadoug
Copy link
Copy Markdown
Contributor Author

@felixcheung @echarles does that explain a bit better?

@felixcheung
Copy link
Copy Markdown
Member

Ah that makes perfect sense, thanks for the detailed explanation.
LGTM.

@corneadoug corneadoug force-pushed the improve/flushStreamingOutput branch from e871ba6 to 76a66a4 Compare April 8, 2016 08:03
@asfgit asfgit closed this in cf34b68 Apr 11, 2016
@corneadoug corneadoug deleted the improve/flushStreamingOutput branch April 11, 2016 02:00
onkarshedge pushed a commit to onkarshedge/incubator-zeppelin that referenced this pull request May 11, 2016
### What is this PR for?
This PR is to improve the result output when having interpreter streaming running.
If you navigate outside the notebook, then come back inside, the streaming output will append below previous result.

### What type of PR is it?
Improvement

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-772

### How should this be tested?
You can run this
```
(1 to 40).foreach{ i=>
    Thread.sleep(1000)
    println("Hello " + i)
}
```
Then navigate to home, then back to the notebook.
You should see the newly streamed result only.

### Screenshots
Before:
![before](https://cloud.githubusercontent.com/assets/710411/14101710/2b3608f6-f5d1-11e5-9501-e9ba7b0bc16c.gif)

After:
![after](https://cloud.githubusercontent.com/assets/710411/14101714/31aaab06-f5d1-11e5-8fc1-645a3324a65e.gif)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Damien CORNEAU <[email protected]>

Closes apache#804 from corneadoug/improve/flushStreamingOutput and squashes the following commits:

76a66a4 [Damien CORNEAU] Flush on first text append event
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants