Skip to content

drive API: file upload stream backpressure issue #1354

@FranckFreiburger

Description

@FranckFreiburger

It seems that upload stream backpressure is not handled properly.
In the following example, the stream is consumed very quickly (100% reached in 2 seconds) whereas the upload take a while... (that is expected with my ADSL connexion). This mean that the whole file is loaded into memory then sent gradually, which disables the benefit of the streams.

Using your example:

  var fileName = 'ReactOS-0.4.3-iso.zip'; // test file, ~100MB
  const fileSize = fs.statSync(fileName).size;
  const res = await drive.files.create(
    {
      requestBody: {
        // a requestBody element is required if you want to use multipart
      },
      media: {
        body: fs.createReadStream(fileName),
      },
    },
    {
      // Use the `onUploadProgress` event from Axios to track the
      // number of bytes uploaded to this point.
      onUploadProgress: evt => {
        const progress = (evt.bytesRead / fileSize) * 100;
        process.stdout.clearLine();
        process.stdout.cursorTo(0);
        process.stdout.write(`${Math.round(progress)}% complete`);
      },
    }
  );
  console.log(res.data);

Configuration:
Node.js v10.10.0
googleapis 33.0.0
OS windows7

Metadata

Metadata

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions