Skip to content

MongoDB find memory leak #10315

@FxEasy

Description

@FxEasy

Issue description

MongoDB leaks memory whenever find or similar functions are called.

Expected Behavior

TypeORM should not leak cursors into memory after executing requests.

Actual Behavior

Using the find, findBy, and findAndCount with mongodb causes memory leaks, with especially high frequency applications & large data sets the memory leaks get much worse. It leaks FindCursor along with ClientSession for the most part.

Steps to reproduce

import { ObjectId } from "mongodb";
import { Entity, ObjectIdColumn, DataSource } from "typeorm";
import "reflect-metadata";

@Entity()
export class User {
    @ObjectIdColumn()
    user: ObjectId;
}

async function main() {
    const ds = new DataSource({
        type: "mongodb",
        url: "***",
        synchronize: true,
        useNewUrlParser: true,
        useUnifiedTopology: true,
        entities: [User]
    })
    await ds.initialize();
    const userRepository = ds.getMongoRepository(User);
    setInterval(async () => {
        await userRepository.createEntityCursor({}).toArray();
    }, 10);
}

main().catch(err => console.error(err));

Here we never close the cursor, this is done within the typeorm source code in multiple places; the cursor is also hidden so a user cannot simply close it after retrieving the results.

Replacing createEntityCursor({}).toArray() with findAndCount({}) causes the same leak which is uncontrollable to the end user.

My Environment

Dependency Version
Operating System Arch Linux
Node.js version 18.17.1
Typescript version ^5.2.2
TypeORM version ^0.3.17

Additional Context

https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1189
https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1149
https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1109

Some places where the entity cursor is used in a similar way to the reproduction. Even worse, hiding the result in a Promise<Entity[]> neglects the user of the ability to close the cursor once complete.

It seems that createEntityCursor will leak memory faster especially dependent on the size of the schema.

It also looks like createCursor doesn't emit the same behavior. This sort of leads me to believe it's apart of the clone we're running here: https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1013 which isn't actually being cleaned up in the "normal" way.

Relevant Database Driver(s)

  • aurora-mysql
  • aurora-postgres
  • better-sqlite3
  • cockroachdb
  • cordova
  • expo
  • mongodb
  • mysql
  • nativescript
  • oracle
  • postgres
  • react-native
  • sap
  • spanner
  • sqlite
  • sqlite-abstract
  • sqljs
  • sqlserver

Are you willing to resolve this issue by submitting a Pull Request?

Yes, I have the time, but I don't know how to start. I would need guidance.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions