Skip to content

Commit 0f8de5e

Browse files
committed
stream: Simplify flowing, passive data listening
Closes #5860 In streams2, there is an "old mode" for compatibility. Once switched into this mode, there is no going back. With this change, there is a "flowing mode" and a "paused mode". If you add a data listener, then this will start the flow of data. However, hitting the `pause()` method will switch *back* into a non-flowing mode, where the `read()` method will pull data out. Every time `read()` returns a data chunk, it also emits a `data` event. In this way, a passive data listener can be added, and the stream passed off to some other reader, for use with progress bars and the like. There is no API change beyond this added flexibility.
1 parent 5fcd6e4 commit 0f8de5e

11 files changed

Lines changed: 433 additions & 201 deletions

doc/api/stream.markdown

Lines changed: 60 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -104,11 +104,35 @@ Readable stream.
104104
A Readable stream will not start emitting data until you indicate that
105105
you are ready to receive it.
106106

107-
Readable streams have two "modes": a **flowing mode** and a **non-flowing
107+
Readable streams have two "modes": a **flowing mode** and a **paused
108108
mode**. When in flowing mode, data is read from the underlying system
109-
and provided to your program as fast as possible. In non-flowing
110-
mode, you must explicitly call `stream.read()` to get chunks of data
111-
out.
109+
and provided to your program as fast as possible. In paused mode, you
110+
must explicitly call `stream.read()` to get chunks of data out.
111+
Streams start out in paused mode.
112+
113+
**Note**: If no data event handlers are attached, and there are no
114+
[`pipe()`][] destinations, and the stream is switched into flowing
115+
mode, then data will be lost.
116+
117+
You can switch to flowing mode by doing any of the following:
118+
119+
* Adding a [`'data'` event][] handler to listen for data.
120+
* Calling the [`resume()`][] method to explicitly open the flow.
121+
* Calling the [`pipe()`][] method to send the data to a [Writable][].
122+
123+
You can switch back to paused mode by doing either of the following:
124+
125+
* If there are no pipe destinations, by calling the [`pause()`][]
126+
method.
127+
* If there are pipe destinations, by removing any [`'data'` event][]
128+
handlers, and removing all pipe destinations by calling the
129+
[`unpipe()`][] method.
130+
131+
Note that, for backwards compatibility reasons, removing `'data'`
132+
event handlers will **not** automatically pause the stream. Also, if
133+
there are piped destinations, then calling `pause()` will not
134+
guarantee that the stream will *remain* paused once those
135+
destinations drain and ask for more data.
112136

113137
Examples of readable streams include:
114138

@@ -144,9 +168,9 @@ again when more data is available.
144168

145169
* `chunk` {Buffer | String} The chunk of data.
146170

147-
If you attach a `data` event listener, then it will switch the stream
148-
into flowing mode, and data will be passed to your handler as soon as
149-
it is available.
171+
Attaching a `data` event listener to a stream that has not been
172+
explicitly paused will switch the stream into flowing mode. Data will
173+
then be passed as soon as it is available.
150174

151175
If you just want to get all the data out of the stream as fast as
152176
possible, this is the best way to do so.
@@ -200,9 +224,9 @@ bytes. If `size` bytes are not available, then it will return `null`.
200224
If you do not specify a `size` argument, then it will return all the
201225
data in the internal buffer.
202226

203-
This method should only be called in non-flowing mode. In
204-
flowing-mode, this method is called automatically until the internal
205-
buffer is drained.
227+
This method should only be called in paused mode. In flowing mode,
228+
this method is called automatically until the internal buffer is
229+
drained.
206230

207231
```javascript
208232
var readable = getReadableStreamSomehow();
@@ -214,6 +238,9 @@ readable.on('readable', function() {
214238
});
215239
```
216240

241+
If this method returns a data chunk, then it will also trigger the
242+
emission of a [`'data'` event][].
243+
217244
#### readable.setEncoding(encoding)
218245

219246
* `encoding` {String} The encoding to use.
@@ -244,9 +271,9 @@ readable.on('data', function(chunk) {
244271
This method will cause the readable stream to resume emitting `data`
245272
events.
246273

247-
This method will switch the stream into flowing-mode. If you do *not*
274+
This method will switch the stream into flowing mode. If you do *not*
248275
want to consume the data from a stream, but you *do* want to get to
249-
its `end` event, you can call `readable.resume()` to open the flow of
276+
its `end` event, you can call [`readable.resume()`][] to open the flow of
250277
data.
251278

252279
```javascript
@@ -259,13 +286,9 @@ readable.on('end', function(chunk) {
259286

260287
#### readable.pause()
261288

262-
This method will cause a stream in flowing-mode to stop emitting
263-
`data` events. Any data that becomes available will remain in the
264-
internal buffer.
265-
266-
This method is only relevant in flowing mode. When called on a
267-
non-flowing stream, it will switch into flowing mode, but remain
268-
paused.
289+
This method will cause a stream in flowing mode to stop emitting
290+
`data` events, switching out of flowing mode. Any data that becomes
291+
available will remain in the internal buffer.
269292

270293
```javascript
271294
var readable = getReadableStreamSomehow();
@@ -414,7 +437,7 @@ entire Streams API as it is today. (See "Compatibility" below for
414437
more information.)
415438

416439
If you are using an older Node library that emits `'data'` events and
417-
has a `pause()` method that is advisory only, then you can use the
440+
has a [`pause()`][] method that is advisory only, then you can use the
418441
`wrap()` method to create a [Readable][] stream that uses the old stream
419442
as its data source.
420443

@@ -1298,23 +1321,23 @@ simpler, but also less powerful and less useful.
12981321
events would start emitting immediately. If you needed to do some
12991322
I/O to decide how to handle data, then you had to store the chunks
13001323
in some kind of buffer so that they would not be lost.
1301-
* The `pause()` method was advisory, rather than guaranteed. This
1324+
* The [`pause()`][] method was advisory, rather than guaranteed. This
13021325
meant that you still had to be prepared to receive `'data'` events
13031326
even when the stream was in a paused state.
13041327

13051328
In Node v0.10, the Readable class described below was added. For
13061329
backwards compatibility with older Node programs, Readable streams
13071330
switch into "flowing mode" when a `'data'` event handler is added, or
1308-
when the `pause()` or `resume()` methods are called. The effect is
1309-
that, even if you are not using the new `read()` method and
1310-
`'readable'` event, you no longer have to worry about losing `'data'`
1311-
chunks.
1331+
when the [`resume()`][] method is called. The effect is that, even if
1332+
you are not using the new `read()` method and `'readable'` event, you
1333+
no longer have to worry about losing `'data'` chunks.
13121334

13131335
Most programs will continue to function normally. However, this
13141336
introduces an edge case in the following conditions:
13151337

1316-
* No `'data'` event handler is added.
1317-
* The `pause()` and `resume()` methods are never called.
1338+
* No [`'data'` event][] handler is added.
1339+
* The [`resume()`][] method is never called.
1340+
* The stream is not piped to any writable destination.
13181341

13191342
For example, consider the following code:
13201343

@@ -1336,7 +1359,7 @@ simply discarded. However, in Node v0.10 and beyond, the socket will
13361359
remain paused forever.
13371360

13381361
The workaround in this situation is to call the `resume()` method to
1339-
trigger "old mode" behavior:
1362+
start the flow of data:
13401363

13411364
```javascript
13421365
// Workaround
@@ -1352,9 +1375,9 @@ net.createServer(function(socket) {
13521375
}).listen(1337);
13531376
```
13541377

1355-
In addition to new Readable streams switching into flowing-mode, pre-v0.10
1356-
style streams can be wrapped in a Readable class using the `wrap()`
1357-
method.
1378+
In addition to new Readable streams switching into flowing mode,
1379+
pre-v0.10 style streams can be wrapped in a Readable class using the
1380+
`wrap()` method.
13581381

13591382

13601383
### Object Mode
@@ -1494,3 +1517,9 @@ modify them.
14941517
[_write]: #stream_writable_write_chunk_encoding_callback_1
14951518
[`util.inherits`]: util.html#util_util_inherits_constructor_superconstructor
14961519
[`end()`]: #stream_writable_end_chunk_encoding_callback
1520+
[`'data'` event]: #stream_event_data
1521+
[`resume()`]: #stream_readable_resume
1522+
[`readable.resume()`]: #stream_readable_resume
1523+
[`pause()`]: #stream_readable_pause
1524+
[`unpipe()`]: #stream_readable_unpipe_destination
1525+
[`pipe()`]: #stream_readable_pipe_destination_options

0 commit comments

Comments
 (0)