-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
Description
Issue description
MongoDB leaks memory whenever find or similar functions are called.
Expected Behavior
TypeORM should not leak cursors into memory after executing requests.
Actual Behavior
Using the find, findBy, and findAndCount with mongodb causes memory leaks, with especially high frequency applications & large data sets the memory leaks get much worse. It leaks FindCursor along with ClientSession for the most part.
Steps to reproduce
import { ObjectId } from "mongodb";
import { Entity, ObjectIdColumn, DataSource } from "typeorm";
import "reflect-metadata";
@Entity()
export class User {
@ObjectIdColumn()
user: ObjectId;
}
async function main() {
const ds = new DataSource({
type: "mongodb",
url: "***",
synchronize: true,
useNewUrlParser: true,
useUnifiedTopology: true,
entities: [User]
})
await ds.initialize();
const userRepository = ds.getMongoRepository(User);
setInterval(async () => {
await userRepository.createEntityCursor({}).toArray();
}, 10);
}
main().catch(err => console.error(err));Here we never close the cursor, this is done within the typeorm source code in multiple places; the cursor is also hidden so a user cannot simply close it after retrieving the results.
Replacing createEntityCursor({}).toArray() with findAndCount({}) causes the same leak which is uncontrollable to the end user.
My Environment
| Dependency | Version |
|---|---|
| Operating System | Arch Linux |
| Node.js version | 18.17.1 |
| Typescript version | ^5.2.2 |
| TypeORM version | ^0.3.17 |
Additional Context
https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1189
https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1149
https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1109
Some places where the entity cursor is used in a similar way to the reproduction. Even worse, hiding the result in a Promise<Entity[]> neglects the user of the ability to close the cursor once complete.
It seems that createEntityCursor will leak memory faster especially dependent on the size of the schema.
It also looks like createCursor doesn't emit the same behavior. This sort of leads me to believe it's apart of the clone we're running here: https://github.com/typeorm/typeorm/blob/master/src/entity-manager/MongoEntityManager.ts#L1013 which isn't actually being cleaned up in the "normal" way.
Relevant Database Driver(s)
- aurora-mysql
- aurora-postgres
- better-sqlite3
- cockroachdb
- cordova
- expo
- mongodb
- mysql
- nativescript
- oracle
- postgres
- react-native
- sap
- spanner
- sqlite
- sqlite-abstract
- sqljs
- sqlserver
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, but I don't know how to start. I would need guidance.