add @@asyncIterator to ReadableStreamDefaultReader#950
add @@asyncIterator to ReadableStreamDefaultReader#950devsnek wants to merge 3 commits intowhatwg:masterfrom devsnek:feature/defaultreader-asynciterator
Conversation
index.bs
Outdated
| </tr> | ||
| </table> | ||
|
|
||
| <h5 id="default-reader-asynciterator-prototype-next" method for="ReadableStreamDefaultReaderAsyncIteratorPrototype"> |
There was a problem hiding this comment.
This and return() should be <h4>'s.
index.bs
Outdated
| </emu-alg> | ||
|
|
||
| <h5 id="default-reader-iterator" method for="ReadableStreamDefaultReader"> | ||
| iterator({ cancel = true } = {}) |
There was a problem hiding this comment.
There is no precedent for an iterator method anywhere in the platform and I'm reluctant to start that now. Just having @@asyncIterator is good enough IMO. (@@asyncIterator can take a parameter too.)
There was a problem hiding this comment.
iterator() came from the linked issue, i'm unsure what to do here.
There was a problem hiding this comment.
Yeah we should have this method. But we should flip it to preventCancel = true. https://w3ctag.github.io/design-principles/#prefer-dict-to-bool third paragraph.
There was a problem hiding this comment.
How about "getIterator" or "iterate"? The first provides consistency with the existing "getReader". I do prefer a name with a verb in it.
There was a problem hiding this comment.
@domenic The default for this would be preventCancel = false, right?
There was a problem hiding this comment.
Hmm, now I'm thinking "values()" for symmetry with sync iterators?
There was a problem hiding this comment.
OTOH that has the potential to be confused as a sync iterator…
index.bs
Outdated
|
|
||
| <emu-alg> | ||
| 1. Let _O_ be *this* value. | ||
| 1. If Type(_O_) is not Object, throw a *TypeError* exception. |
There was a problem hiding this comment.
- This needs to check if O has a [[Reader]] internal slot. Otherwise O might not be a ReadableStreamDefaultReaderAsyncIterator. It might be worthwhile to have an IsReadableStreamDefaultReaderAsyncIterator abstract operation.
- This spec uses this rather "this value" (the ECMA-262 convention). So you could just do Type(this).
Ditto below for cancel.
index.bs
Outdated
| <emu-alg> | ||
| 1. Let _O_ be *this* value. | ||
| 1. If Type(_O_) is not Object, throw a *TypeError* exception. | ||
| 1. Let _reader_ be O.[[Reader]]. |
index.bs
Outdated
| 1. Let _O_ be *this* value. | ||
| 1. If Type(_O_) is not Object, throw a *TypeError* exception. | ||
| 1. Let _reader_ be O.[[Reader]]. | ||
| 1. Let _read_ be ? GetMethod(_reader_, "read"). |
There was a problem hiding this comment.
"read"
Ditto below for cancel.
index.bs
Outdated
| 1. If Type(_O_) is not Object, throw a *TypeError* exception. | ||
| 1. Let _reader_ be O.[[Reader]]. | ||
| 1. Let _read_ be ? GetMethod(_reader_, "read"). | ||
| 1. Return ? Call(_read_, _reader_); |
index.bs
Outdated
|
|
||
| <h3 id="default-reader-asynciterator-prototype" interface lt="ReadableStreamDefaultReaderAsyncIteratorPrototype"> | ||
| ReadableStreamDefaultReaderAsyncIteratorPrototype | ||
| </h3> |
There was a problem hiding this comment.
It should be written somewhere that this inherits from %AsyncIteratorPrototype%.
There was a problem hiding this comment.
yea ik... i wasn't sure how/where to do that.
There was a problem hiding this comment.
is this a must for implementers? Or is it how things are spec'ed?
There was a problem hiding this comment.
@mcollina all async iterators should inherit from (have [[Prototype]] set to) %AsyncIteratorPrototype%, but it doesn't happen by default. i added it in my last commit.
There was a problem hiding this comment.
@devsnek reading from https://tc39.github.io/ecma262/#sec-asynciteratorprototype, it does not seem to be mandatory for implementers. I'm just trying to understand if there are any benefits in doing so.
There was a problem hiding this comment.
@mcollina Yes, it is mandatory; and yes, there are benefits. Having it would allow
for await (const chunk of stream.getReader().getIterator()) {
}in addition to just
for await (const chunk of stream.getReader()) {
}This is fully consistent with how sync iterators operate, that you can do Array.from(map.entries()) in addition to just Array.from(map).
Additionally, it would allow ECMA-262 to extend %AsyncIteratorPrototype% and the improvements would directly reflect on streams.
There was a problem hiding this comment.
Also, you can get the %AsyncIteratorPrototype% through as you have found out.Object.getPrototypeOf(Object.getPrototypeOf((async function* () {}).prototype)).
index.bs
Outdated
| </h5> | ||
|
|
||
| <div class="note"> | ||
| The <code>iterator</code> method returns an async iterator which can be used to consume the stream. The return method |
index.bs
Outdated
|
|
||
| <div class="note"> | ||
| The <code>iterator</code> method returns an async iterator which can be used to consume the stream. The return method | ||
| of this iterator will optionally release or cancel this reader. |
index.bs
Outdated
|
|
||
| <h3 id="default-reader-asynciterator-prototype" interface lt="ReadableStreamDefaultReaderAsyncIteratorPrototype"> | ||
| ReadableStreamDefaultReaderAsyncIteratorPrototype | ||
| </h3> |
There was a problem hiding this comment.
is this a must for implementers? Or is it how things are spec'ed?
| if (O._cancel === true) { | ||
| const reader = O._reader; | ||
| const cancel = reader.cancel; | ||
| cancel.call(reader); |
There was a problem hiding this comment.
I think it should be return cancel.call(reader);
There was a problem hiding this comment.
The spec says "Perform ? Call(…)". There's no return anywhere.
| cancel.call(reader); | ||
| } | ||
| } | ||
| }, Object.getPrototypeOf(Object.getPrototypeOf(async function* () {}).prototype)); |
There was a problem hiding this comment.
What benefit does it add setting the prototype to this?
index.bs
Outdated
|
|
||
| <emu-alg> | ||
| 1. Let _iterator_ be ? GetMethod(*this*, "iterator"). | ||
| 1. Return ? Call(_iterator_, *this*). |
There was a problem hiding this comment.
This is not idiomatic and not how existing iterators work in ECMAScript and Web IDL. It in fact should be equal to the iterator method (or whatever it ends up being called). See how [].values === [][Symbol.iterator], Map.prototype.entries === Map.prototype[Symbol.iterator], and URLSearchParams.prototype.entries === URLSearchParams.prototype[Symbol.iterator].
There was a problem hiding this comment.
Yeah for spec style we should follow https://tc39.github.io/ecma262/#sec-map.prototype-@@iterator I guess.
There was a problem hiding this comment.
agreed, i just wasn't sure how to write that out in bs. thanks for the link.
| if (O._cancel === true) { | ||
| const reader = O._reader; | ||
| const cancel = reader.cancel; | ||
| cancel.call(reader); |
There was a problem hiding this comment.
This does not implement GetMethod fully. Additionally I'd prefer using Reflect.apply().
There was a problem hiding this comment.
We should put a full GetMethod implementation in helpers.js.
|
|
||
| const ReadableStreamDefaultReaderAsyncIteratorPrototype = Object.setPrototypeOf({ | ||
| _reader: undefined, | ||
| _cancel: undefined, |
There was a problem hiding this comment.
Why does the prototype have these properties? The prototype is not an instance of ReadableStreamDefaultReaderAsyncIterator.
domenic
left a comment
There was a problem hiding this comment.
Great start, and sorry for the delay in reviewing!
index.bs
Outdated
|
|
||
| <emu-alg> | ||
| 1. Let _iterator_ be ? GetMethod(*this*, "iterator"). | ||
| 1. Return ? Call(_iterator_, *this*). |
There was a problem hiding this comment.
Yeah for spec style we should follow https://tc39.github.io/ecma262/#sec-map.prototype-@@iterator I guess.
index.bs
Outdated
|
|
||
| <h3 id="default-reader-asynciterator-prototype" interface lt="ReadableStreamDefaultReaderAsyncIteratorPrototype"> | ||
| ReadableStreamDefaultReaderAsyncIteratorPrototype | ||
| </h3> |
There was a problem hiding this comment.
Style nit: this introduces spaces between the > and R and the e and the <. No good. Same throughout various places in the patch.
index.bs
Outdated
| </h4> | ||
|
|
||
| <emu-alg> | ||
| 1. If ! IsReadableStreamDefaultReaderAsyncIteratorPrototype(*this*) is *false*, throw a *TypeError* exception. |
There was a problem hiding this comment.
IsReadableStreamDefaultReaderAsyncIteratorPrototype doesn't seem to be defined?
index.bs
Outdated
|
|
||
| <h4 id="default-reader-asynciterator-prototype-return" method for="ReadableStreamDefaultReaderAsyncIteratorPrototype"> | ||
| return() | ||
| </h4> |
There was a problem hiding this comment.
So apparently async iterator return methods should take a value argument and return a promise for { done: true, value }.
You'll want to return the result of transforming the call to cancel() by a function that creates and returns the { done: true, value } tuple (probably using ReadableStreamCreateReadResult for consistency).
| } | ||
|
|
||
| iterator({ cancel = true } = {}) { | ||
| const O = this; |
There was a problem hiding this comment.
To follow the spec more exactly, don't do the O aliasing. Here and everywhere below.
index.bs
Outdated
| <emu-alg> | ||
| 1. If ! IsReadableStreamDefaultReader(*this*) is *false*, throw a *TypeError* exception. | ||
| 1. If *this*.[[ownerReadableStream]] is *undefined*, throw a *TypeError* exception. | ||
| 1. Let _iterator_ be ? ObjectCreate(`<a idl>ReadableStreamDefaultReaderAsyncIteratorPrototype</a>`, « [[Reader]], |
There was a problem hiding this comment.
This should be !, not ?, I think.
| if (O._cancel === true) { | ||
| const reader = O._reader; | ||
| const cancel = reader.cancel; | ||
| cancel.call(reader); |
There was a problem hiding this comment.
We should put a full GetMethod implementation in helpers.js.
index.bs
Outdated
| </h4> | ||
|
|
||
| <emu-alg> | ||
| 1. If ! IsReadableStreamDefaultReaderAsyncIteratorPrototype(*this*) is *false*, throw a *TypeError* exception. |
There was a problem hiding this comment.
next() and return() should reject the returned promise instead of throwing an exception. This also means turning any exception thrown from GetMethod and Call into rejections.
index.bs
Outdated
| </emu-alg> | ||
|
|
||
| <h5 id="default-reader-iterator" method for="ReadableStreamDefaultReader"> | ||
| iterator({ cancel = true } = {})</h5> |
There was a problem hiding this comment.
Just don't worry about the line length and put it on the same line as <h5>.
index.bs
Outdated
|
|
||
| <h5 id="default-reader-@@asynciterator" for="ReadableStreamDefaultReader">[@@asyncIterator]()</h5> | ||
|
|
||
| <div class="note"> |
index.bs
Outdated
| </div> | ||
|
|
||
| The initial value of the <code>@@asyncIterator</code> method is the same function object as the initial value of the | ||
| <code>iterator</code> method. |
There was a problem hiding this comment.
This should be linked. Also it should be made explicit which iterator method we are talking about (which object it belongs to), if we have more than one in the future.
index.bs
Outdated
| </emu-alg> | ||
|
|
||
| <h5 id="default-reader-iterator" method for="ReadableStreamDefaultReader"> | ||
| iterator({ cancel = true } = {})</h5> |
There was a problem hiding this comment.
My point about the naming remains. I still would like to see it as getIterator() or iterate().
index.bs
Outdated
| <h5 id="default-reader-iterator" method for="ReadableStreamDefaultReader"> | ||
| iterator({ cancel = true } = {})</h5> | ||
|
|
||
| <div class="note"> |
| <var>x</var> )</h4> | ||
|
|
||
| <emu-alg> | ||
| 1. If Type(_x_) is not Object, return *false*. |
There was a problem hiding this comment.
Per ECMAScript convention, objects usually use a single uppercase letter or a camel-cased phrase as variable name. In this case I believe argument would suffice.
index.bs
Outdated
|
|
||
| <emu-alg> | ||
| 1. If Type(_x_) is not Object, return *false*. | ||
| 1. If _x_ does not have a [[Reader]] and [[Cancel]] internal slots, return *false*. |
index.bs
Outdated
| </emu-alg> | ||
|
|
||
| <h4 id="is-readable-stream-default-reader-asynciterator-prototype"> | ||
| aoid="IsReadableStreamDefaultReaderAsyncIteratorPrototype" nothrow>IsReadableStreamDefaultReaderAsyncIteratorPrototype ( |
There was a problem hiding this comment.
You are testing for if an object is an AsyncIterator, not if it's an object.
index.bs
Outdated
| 1. If ! IsReadableStreamDefaultReaderAsyncIteratorPrototype(*this*) is *false*, throw a *TypeError* exception. | ||
| 1. Let _reader_ be *this*.[[Reader]]. | ||
| 1. If *this*.[[Cancel]] is *true*, then: | ||
| 1. Let _cancel_ be ? GetMethod(_reader_, `"cancel"`). |
There was a problem hiding this comment.
You should wrap this ? in a promise too.
index.bs
Outdated
| </thead> | ||
| <tr> | ||
| <td>\[[Prototype]] | ||
| <td>%AsyncIteratorPrototype% |
There was a problem hiding this comment.
Any reason why this isn't in a .non-normative?
MattiasBuelens
left a comment
There was a problem hiding this comment.
A few typos to fix. Also wondering whether we should use the Call helper function instead of Reflect.apply.
| try { | ||
| const reader = this._reader; | ||
| const read = GetMethod(reader, 'read'); | ||
| return Reflect.apply(read, reader, []); |
There was a problem hiding this comment.
Should we use the Call helper function here? The reference implementation doesn't seem to use Reflect anywhere else, although I'm not sure whether that's a design goal.
index.bs
Outdated
| 1. Let _read_ be GetMethod(_reader_, `"read"`). | ||
| 1. If _read_ is an abrupt completion, return <a>a promise rejected with</a> _read_.[[Value]]. | ||
| 1. Let _result_ be Call(_read_, _reader_). | ||
| 1. If _result_ is an abrupt completion, return <a>a promise rejected with<a/> _result_.[[Value]]. |
There was a problem hiding this comment.
Typo: change <a/> to </a>
index.bs
Outdated
| <a href="#default-reader-read">read</a>() | ||
| <a href="#default-reader-release-lock">releaseLock</a>() | ||
|
|
||
| <a href="#default-reader-getiterator">getIterator</a>({ preventClose = false }) |
There was a problem hiding this comment.
Typo: change preventClose to preventCancel. Same thing for the next line.
index.bs
Outdated
|
|
||
| <emu-alg> | ||
| 1. If Type(_argument_) is not Object, return *false*. | ||
| 1. If _argument_ does not have a [[Reader]] and [[PreventCancel]] internal slots, return *false*. |
There was a problem hiding this comment.
@TimothyGu previously reported this, but it hasn't been tackled yet:
The "a" seems misplaced.
|
I don't understand why a reader object is needed in order to iterate over a stream. I was hoping to be able to write for await (const chunk of stream) { ... }and have it do what I mean. |
|
I can't remember why I thought using the reader as the entry point was a better idea than using the stream. Maybe explicitness or something? But yeah, probably we should switch it... |
|
Does this work? When @@asyncIterator of ReadableStream is called when _arguments_, do:
1. Let _getReader_ be ? GetMethod(*this*, `"getReader"`).
2. Let _reader_ be ? Call(_getReader_, *this*).
3. Let _getIterator_ be ? GetMethod(_reader_, @@asyncIterator).
4. Let _iter_ be ? Call(_getIterator_, _reader_, _arguments_).
5. Return _iter_. |
|
@devsnek That could work... assuming we only add If we were to also add const iterator = readable.getIterator({ preventCancel: true });
iterator.return();
// readable.locked === true;Because What should happen in this case? We could call const iterator = readable.getIterator({ preventCancel: true });
const readPromise = iterator.next();
iterator.return(); // can't unlock, since read is still pending... what now?I don't know if this is a case we want to support. Perhaps we don't need this, and we only have |
|
@MattiasBuelens i'm honestly not a fan of implicitly acquiring a reader for exactly those reasons. One thing though -- This feature would depend on ReadableStreamDefaultBYOBReader having an @@asynciterator, which it does not yet. I think it might be better to hold off on ReadableStream[@@asynciterator] until byob reader has @@asynciterator as well. |
|
@devsnek The default Reader for a ReadableByteStream is still a ReadableStreamDefaultReader, so we don't need to block on BYOB support. I can't see a way to do bring-your-own-buffer async iterators, as there's no way to supply a buffer to the iterator. iterator.return(); // can't unlock, since read is still pending... what now?Can we throw in this case? IIUC, it won't happen when using the |
domenic
left a comment
There was a problem hiding this comment.
First round of review. Here I address superficial issues. I'll upload a patch fixing all of these as I go, as penance for the long turnaround times so far.
index.bs
Outdated
| <p class="note"> | ||
| The <code>getIterator</code> method returns an async iterator which can be used to consume the stream. The | ||
| {{ReadableStreamDefaultReaderAsyncIteratorPrototype/return()}} method of this iterator object will optionally | ||
| {{ReadableStreamDefaultReader/cancel()}} this reader. |
There was a problem hiding this comment.
We should talk about canceling the stream, not link to the public API on reader that does so.
We should also mention the auto-release behavior.
index.bs
Outdated
| 1. If ! IsReadableStreamDefaultReader(*this*) is *false*, throw a *TypeError* exception. | ||
| 1. If *this*.[[ownerReadableStream]] is *undefined*, throw a *TypeError* exception. | ||
| 1. Let _iterator_ be ObjectCreate(`<a idl>ReadableStreamDefaultReaderAsyncIteratorPrototype</a>`, « [[Reader]], | ||
| [[PreventCancel]] »). |
There was a problem hiding this comment.
Interesting. Elsewhere in the spec we fail to add the internal slots. For consistency let's remove that here, and file a separate issue to fix it. (Although I'd like a fix that is less verbose and annoying than the ES spec's repetition of the internal slots list.)
index.bs
Outdated
| <a href="#default-reader-read">read</a>() | ||
| <a href="#default-reader-release-lock">releaseLock</a>() | ||
|
|
||
| <a href="#default-reader-getiterator">getIterator</a>({ preventCancel = false }) |
There was a problem hiding this comment.
In both the algorithm and the header, we should follow the pattern of pipeTo (and other web APIs), of not defaulting to false, but instead doing ToBoolean on the argument.
index.bs
Outdated
| 1. Let _iterator_ be ObjectCreate(`<a idl>ReadableStreamDefaultReaderAsyncIteratorPrototype</a>`, « [[Reader]], | ||
| [[PreventCancel]] »). | ||
| 1. Set _iterator_.[[Reader]] to *this*. | ||
| 1. Set _iterator_.[[PreventCancel]] to ! ToBoolean(_cancel_). |
index.bs
Outdated
| <emu-alg> | ||
| 1. If ! IsReadableStreamDefaultReader(*this*) is *false*, throw a *TypeError* exception. | ||
| 1. If *this*.[[ownerReadableStream]] is *undefined*, throw a *TypeError* exception. | ||
| 1. Let _iterator_ be ObjectCreate(`<a idl>ReadableStreamDefaultReaderAsyncIteratorPrototype</a>`, « [[Reader]], |
There was a problem hiding this comment.
ObjectCreate should not fail ,so add !
index.bs
Outdated
| 1. Let _reader_ be *this*.[[Reader]]. | ||
| 1. Let _read_ be GetMethod(_reader_, `"read"`). | ||
| 1. If _read_ is an abrupt completion, return <a>a promise rejected with</a> _read_.[[Value]]. | ||
| 1. Let _result_ be Call(_read_, _reader_). |
index.bs
Outdated
| 1. If _cancel_ is an abrupt completion, return <a>a promise rejected with</a> _cancel_.[[Value]]. | ||
| 1. Let _result_ be Call(_cancel_.[[Value]], _reader_, « _value_ »). | ||
| 1. If _result_ is an abrupt completion, return <a>a promise rejected with</a> _result_.[[Value]]. | ||
| 1. Return <a>a promise resolved with</a> ? ReadableStreamCreateReadResult(_value_, *true*, *true*). |
There was a problem hiding this comment.
ReadableStreamCreateReadResult cannot fail, so use !
index.bs
Outdated
|
|
||
| <emu-alg> | ||
| 1. If Type(_argument_) is not Object, return *false*. | ||
| 1. If _argument_ does not have [[Reader]] and [[PreventCancel]] internal slots, return *false*. |
There was a problem hiding this comment.
We only generally test one slot.
index.bs
Outdated
| <td class="non-normative">%AsyncIteratorPrototype% | ||
| </tr> | ||
| <tr> | ||
| <td>\[[Reader]] |
There was a problem hiding this comment.
For no good reason, we use lowercase slot names in this spec. We have an open bug to fix, but we should align until then.
index.bs
Outdated
| <td class="non-normative">%AsyncIteratorPrototype% | ||
| </tr> | ||
| <tr> | ||
| <td>\[[Reader]] |
There was a problem hiding this comment.
We should have a slot name that's more distinguishable for the brand check.
|
OK, here are the big issues left, from what I can see:
|
| 1. If _read_ is an abrupt completion, return <a>a promise rejected with</a> _read_.[[Value]]. | ||
| 1. Let _result_ be Call(_read_.[[Value]], _reader_). | ||
| 1. If _result_ is an abrupt completion, return <a>a promise rejected with</a> _result_.[[Value]]. | ||
| 1. Return <a>a promise resolved with</a> _result_.[[Value]]. |
There was a problem hiding this comment.
This is missing auto-release if result.[[Value]]'s done property is set to true, discussed in #778 (comment)
| 1. If _cancel_ is an abrupt completion, return <a>a promise rejected with</a> _cancel_.[[Value]]. | ||
| 1. Let _result_ be Call(_cancel_.[[Value]], _reader_, « _value_ »). | ||
| 1. If _result_ is an abrupt completion, return <a>a promise rejected with</a> _result_.[[Value]]. | ||
| 1. Return <a>a promise resolved with</a> ! ReadableStreamCreateReadResult(_value_, *true*, *true*). |
There was a problem hiding this comment.
This is missing auto-release, discussed in #778 (comment) . It should happen in both branches.
But then what would this do? const reader = stream.getReader();
for await (const chunk of reader) {
break;
}
reader.read();Would the If you want to use a reader first, and then iterate, it seems reasonable to wait for your read and then const reader = stream.getReader();
const { value, done } = await reader.read();
// do something with first chunk
reader.releaseLock();
for await (const chunk of stream) {
// do something with rest of the chunks
}In general, I think having multiple ways to do something tends to lead to confused arguments about which way is "right". So I favour only having the methods on ReadableStream. |
but we would still have to acquire and lock, it would just be behind the scenes. I'm okay with an alias but this should definitely be on the reader be sure that's where the data reading happens. |
I'm not sure I agree with this reasoning. In general, we provide higher-level utilities on the stream, notably pipeTo() and tee(). Under the scenes, those use readers. But they're intentionally meant to allow consumers to not care about the whole reader paradigm. In contrast, when you're using a reader, these high-level conveniences are not interesting or necessary---you're doing chunk-wise manual manipulation. |
Closes #778
probably needs a lot of editorial :)
Preview | Diff