Hello,

I have an issue regarding createReadStream method in nodejs bigtable client, in may situation I need to have a way to get only one item from the stream and do some process on it then get the next and so on, and I don't want to store all the data in memory, and bigtable client in nodejs didn't provide any thing to do it, I use the createReadStream and pause the stream once I get an Item/Row from the table.

This worked fine for small data, but when I tried to do it on large data about 500k rows I didn't retrieve all the rows that I have in the table for the query I made (the 'end' event is emitted before I get all the result from the table).

My code use class similar to this code, the class is supposed to return promise resolves to the next item from the stream or undefined if the stream is finished

export default class Iterable {
    stream: any;
    buffer: any[] = [];
    public cnt: number = 0;
    id = 0;

    constructor(id: number, stream: any) {
        this.id = id;
        this.stream = stream;

        this.stream.on('end', () => {
            console.log('end', this.cnt, this.buffer.length);
            this.buffer.push(undefined);
        });
        this.stream.once('data', (d: any) => {
            this.buffer.push(d);
            this.stream.pause();
        });
    }

    async next(): Promise<any> {
        this.stream.resume();
        return new Promise(resolve => {
            if(this.cnt < this.buffer.length) {
                resolve(this.buffer[this.cnt++]);
            } else {
                let id = setInterval(() => {
                    if (this.cnt < this.buffer.length) {
                        clearInterval(id);
                        resolve(this.buffer[this.cnt++]);
                    }
                }, 1);
            }
        });
    }
}

I also try to use once to only listen on data event on time each time I call next() but I still also didn't get all the data that I should get.

I really hope you solve this issue or create another way to get only one item from the bigtable at each step.

Environment details

• OS: Ubuntu 19.10
• Node.js version: 10.15.2
• npm version: 6.12.0
• @google-cloud/bigtable version: 2.3.1

I also use bigtable emulator

Steps to reproduce

1. Have a big table with large number of rows
2. Try to use the above class to get all the data

Thanks!