Instead of adding those properties later only if the styles have them, we will always have those so we can take advantage of "hidden class"

Remove clean as it changes the "hidden class of the object", it is better to just keep the properties but change their values

No need to create the instance just for this, recreated the logic here

the number conversion can probably be removed

instead of this:

const str = <1 GB of string>

rawParse(str)

the str will be saved in memory until the execution finish

we get a function to get the string so when the function finish the string can be garbage collected

Split the string into chunks of 1MB to allow fast garbage collection of 1MB at a time - instead of holding the reference to the string we split into chunks of 1MB and after processing is finished we can remove the reference so it can be GC

also we split to 1MB so it's not too little that after every processing we gonna change the array size (shift) and not too big that we hold reference to too much data

The shift here is important so we don't have any reference to the data once we finish processing

just a note, but shift is O(n) and causes the contents of the arr to be moved which makes this loop On^2. Depending on the amount of chunks you are expecting, you may be able to squeeze some extra perf by using a linked list

Not necessarily, even though linked list are theoretically better for this they might not be, the case, I tried once to move nodejs webstream internal from using array to linked list and it was actually worst

what we can do instead is set the value to undefined so it can be garbage collected and we won't change the array size

Correct (hence my comment about chunk count). Context is important and I don't have much of it here, I wanted to flag this mostly for awareness.

In many cases when dealing with small iterators, micro-optimizations like these are redundant. What I have seen happen with code like this is that even though you don't see any improvement, someone in the future might run into some obscure edge case (maybe by processing something with an insanely high chunk count) and experience degraded perf, but you are the lib author so you know if this is ever likely to happen (if ever) and why you imo don't need to even handle it.

I wanted to raise this for awareness (sorry if I sidetracked the discussion), but I think I should add that you did a very good job on the entire PR 🙏

you are right, also I'm actually not the lib author

I think it will be okay for sane uses (like 1k chunks), as the reallocating and copying of such an array would be peanuts compared to processing of chunks themselves...

The shift here is important so we don't have any reference to the data once we finish processing

Would be reasonable to add a comment in the code, so it won't trigger whoever will be reading that code.

assigned to undefined and added comment

Because the processing can take time we don't want the user to not do anything with the data until all of it is ready, so this way the user can do something with it

and some use cases just don't need some of the spans, so it can be GCed faster

consider this:

function createSpans() {
	return [{text: 'hello'}, {text: ''}, {text: 'world'}];
}

function run() {
	const allSpans = createSpans()

	for(const span of spans) {
		if(span.text.length === 0) {
			continue; // We don't care for empty spans
		}
	}
}

in this example, the span that we don't care can be GCed after the run function finish

but if we use generators the empty span can be GCed immediately:

function* createSpans() {
	yield {text: 'hello'};
	yield {text: ''};
	yield {text: 'world'};
}

function run() {
	const allSpans = createSpans()

	for(const span of spans) {
		if(span.text.length === 0) {
			continue; // We don't care for empty spans
		}
	}
}

(there is no reference to the empty span)