Skip to content
This repository was archived by the owner on Mar 4, 2026. It is now read-only.
This repository was archived by the owner on Mar 4, 2026. It is now read-only.

HUGE memory leak problem #768

@jakeleventhal

Description

@jakeleventhal

Environment details

  • OS: Mac OS 10.15 Beta (19A573a)
  • Node.js version: v12.10.0
  • npm version: 6.10.3
  • @google-cloud/firestore version: 2.3.0

I'm still experiencing a memory leak potentially related to googleapis/gapic-generator-typescript#661. The following is from my yarn.lock:

"@grpc/grpc-js@^0.5.2":
  version "0.5.2"
  resolved "https://registry.yarnpkg.com/@grpc/grpc-js/-/grpc-js-0.5.2.tgz#8e39876033749e1611fe9fa181d9535077805abe"
  integrity sha512-NE1tP/1AF6BqhLdILElnF7aOBfoky+4ZOdZU/0NmKo2d+9F9QD8zGoElpBk/5BfyQZ3u1Zs+wFbDOFpVUzDx1w==
  dependencies:
    semver "^6.0.0"

"@grpc/proto-loader@^0.5.0", "@grpc/proto-loader@^0.5.1":
  version "0.5.1"
  resolved "https://registry.yarnpkg.com/@grpc/proto-loader/-/proto-loader-0.5.1.tgz#48492b53cdda353110b51a4b02f465974729c76f"
  integrity sha512-3y0FhacYAwWvyXshH18eDkUI40wT/uGio7MAegzY8lO5+wVsc19+1A7T0pPptae4kl7bdITL+0cHpnAPmryBjQ==
  dependencies:
    lodash.camelcase "^4.3.0"
    protobufjs "^6.8.6"

Quick note: because of the grpc problems that keep appearing here, we should probably put grpc version as part of the issue template

So it is clear that I'm actually using the "correct" grpc. My application makes many calls to firestore and I'm experiencing massive memory leaks.
Screen Shot 2019-09-28 at 6 53 37 PM

I'm not actually seeing the error message described previously (MaxListenersExceededWarning), however, the memory usage on my application slowly and steady is increasing. I've taken several heap snapshots from local tests over the course of several hours and when comparing snapshots from a few hours apart, I notice that the top culprits for memory allocations are all related to grpc.

Here you can clearly see that the top 4 culprits for the memory delta between snapshots 12 and 4 are vastly more than everything else
Screen Shot 2019-09-28 at 7 08 24 PM

Here are the contents of each from largest to smallest:

  1. (string): This by far is the fastest growing constructor. As you can see, there are 920k new allocations in the comparison between the two snapshots. Practically every single string here is ""(function anonymous( ) { return function BeginTransactionRequest(p){ if(p)for(var ks=Object.keys(p),i=0;i<ks.length;++i)if(p[ks[i]]!=null) this[ks[i]]=p[ks[i]] } })". And this all comes from message.js:13. This is a a file from the package protobufjs which is a dependency of grpc.

Screen Shot 2019-09-28 at 7 47 17 PM

  1. (system): Essentially everything here seems to be coming from from the same message.js:13.

Screen Shot 2019-09-28 at 7 29 51 PM

I also have tens of thousands of objects that look like this (from grpc)

Screen Shot 2019-09-28 at 8 10 43 PM

  1. (array): Practically every single array allocation here is a alloc size 32 and is linked to type.js:31 and namespace.js:95, both of which are also part of protobufjs.

Screen Shot 2019-09-28 at 8 18 42 PM

"Field" is also related to the same line numbers and is actually directly linked to grpc

Screen Shot 2019-09-28 at 8 20 38 PM

  1. Array: There are 235k of these and they all look the same - Array with an alloc size of 32 with some connection to grpc

Screen Shot 2019-09-28 at 7 12 35 PM

Metadata

Metadata

Labels

api: firestoreIssues related to the googleapis/nodejs-firestore API.externalThis issue is blocked on a bug with the actual product.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions