-
-
Notifications
You must be signed in to change notification settings - Fork 6.5k
feat: add new undefined and null behavior flags #11332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Introduces two new configuration options for find operations: - `treatJsNullAsSqlNull`: Allows treating JavaScript null values as SQL NULL - `throwOnUndefinedInFind`: Enables throwing an error when undefined values are encountered These options provide more granular control over how null and undefined values are handled in query conditions, improving type safety and query flexibility.
300e8f8 to
0345764
Compare
…efined-and-null-flags
…efined-and-null-flags
* fix(undefined-null-flags): handle null for relations * fix: handle undefined * refactor: reorganize if blocks
…efined-and-null-flags
commit: |
|
@coderabbitai summary |
|
Caution Review failedThe pull request is closed. WalkthroughAdds a DataSource option Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Caller
participant DS as DataSource
participant QB as QueryBuilder/Repo
participant WB as WhereBuilder
participant DB as Database
Caller->>DS: create(options with invalidWhereValuesBehavior)
Caller->>QB: find/update/delete({ where })
QB->>WB: buildWhere(where, DS.options)
WB->>WB: for each condition: inspect value
alt value is undefined
opt behavior == "ignore"
WB-->>QB: skip condition
end
opt behavior == "throw"
WB-->>Caller: throw TypeORMError ("Undefined value encountered")
end
else value is null
opt behavior == "ignore"
WB-->>QB: skip condition
end
opt behavior == "sql-null"
WB-->>QB: add "IS NULL" condition
end
opt behavior == "throw"
WB-->>Caller: throw TypeORMError ("Null value encountered")
end
else normal value
WB-->>QB: add equality / parameterized condition
end
QB->>DB: execute generated SQL
DB-->>QB: results / affected rows
QB-->>Caller: return results / ack
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
sgarner
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great work @naorpeled !
Just a couple of things in the docs I think could be tweaked.
gioboa
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @naorpeled 💪💯
This PR adds that as an option, but the types will still prevent known-nullable values at compile time. There are good reasons why the designers of SQL decided that Automatically converting JS When a const usersWithNoEmail = await userRepository.findBy({ email: null });With transformation that would execute: SELECT * FROM user WHERE email IS NULLNo problem. But in other cases you may have a const secrets = await secretRepository.findBy({ projectId: user.projectId });Oops. We forgot that SELECT * FROM secret WHERE project_id IS NULLOh no, that's not what we wanted! We just leaked all the secrets that aren't assigned to any project. The default behavior is potentially even less safe, as the null is ignored entirely: SELECT * FROM secretOh no! We just leaked everyone's secrets! That's a contrived example. But there are often corner cases like this in complex projects which can be dangerous if not properly handled. As a matter of principle, TypeORM should help you to avoid these cases, not help you to shoot yourself in the foot. |
|
Our team recently ran into the exact scenario you're describing: we fetch business accounts where status < some_status and userid = userId. At runtime, however, userId sometimes turns out to be null: Because userId is null, TypeORM omits the condition in the generated SQL, which ends up returning all users with status < some_status. We're currently mitigating it like this: This is safer, but we haven't found a cleaner approach yet beyond adding explicit type checks/validation before calling find() or switching to QueryBuilder for stricter control. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (12)
test/utils/test-utils.ts (1)
488-488: Return a consistent Promise shapeEarly-returning
Promise.resolve()changes the inferred return type toPromise<void | unknown[]>. Prefer returning an empty array for consistency with thePromise.all(...)branch.- if (!connections || connections.length === 0) return Promise.resolve() + if (!connections?.length) return Promise.resolve([])src/data-source/BaseDataSourceOptions.ts (1)
218-236: Broaden JSDoc scope beyond “find operations”This option applies to all where-capable operations (find, query builders, repository/manager update/delete/soft-delete). Update the description to avoid under-scoping.
- /** - * Controls how null and undefined values are handled in find operations. - */ + /** + * Controls how null and undefined values are handled in where conditions across all TypeORM operations + * (e.g. find options, QueryBuilder, repository and entity manager methods). + */docs/docs/data-source/2-data-source-options.md (1)
89-95: Use precise SQL phrasing (“IS NULL” predicate)“Transforms null to SQL NULL” can be misread. Suggest “emits an IS NULL condition” for accuracy.
- - `'sql-null'` - transforms null to SQL NULL + - `'sql-null'` - emits an `IS NULL` conditiondocs/docs/data-source/5-null-and-undefined-handling.md (2)
13-16: Avoid committing to future default changes without consensusThe note states the behavior “will be changing” in future versions. Unless this is an agreed roadmap, soften to a recommendation without asserting a committed change.
-:::note -The current behavior will be changing in future versions of TypeORM, -we recommend setting both `null` and `undefined` behaviors to throw to prepare for these changes -::: +:::note +For maximum safety, we recommend setting both `null` and `undefined` behaviors to `'throw'`. +Future versions may change defaults. +:::
85-98: Align terminology with SQL (“IS NULL”)Mirror the phrasing used elsewhere: “emits an IS NULL condition” rather than “transformed into SQL NULL”.
- JavaScript `null` values are transformed into SQL `NULL` conditions: + JavaScript `null` values emit an `IS NULL` condition:test/functional/null-undefined-handling/entity/Post.ts (1)
21-23: Consider explicit onDelete to match nullable relation intent.Adding onDelete: "SET NULL" ensures FK behavior aligns with Post.category being nullable across drivers.
- @ManyToOne(() => Category, { nullable: true }) + @ManyToOne(() => Category, { nullable: true, onDelete: "SET NULL" })test/functional/null-undefined-handling/query-builders.ts (1)
389-438: Add a nested undefined relation case to lock in semantics.Edge case currently untested: category: { id: undefined } when undefined is 'throw'. Suggest adding to ensure nested handling throws (or is ignored) consistently.
+ it("should throw when nested relation id is undefined and undefined='throw'", async () => { + for (const connection of connections) { + try { + await connection + .createQueryBuilder(Post, "post") + .where({ category: { id: undefined } as any }) + .getMany() + expect.fail("Expected query to throw an error") + } catch (error) { + expect(error).to.be.instanceOf(TypeORMError) + expect(error.message).to.include("Undefined value encountered") + } + } + })test/functional/null-undefined-handling/find-options.ts (1)
354-357: Make message assertions resilient (use include, not exact).Tests currently assert full error strings; minor text changes will cause false negatives. Prefer .to.include on a stable substring.
- expect(error.message).to.equal( - "Undefined value encountered in property 'post.text' of the find operation. Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.", - ) + expect(error.message).to.include("Undefined value encountered") ... - expect(error.message).to.equal( - "Undefined value encountered in property 'Post.text' of the find operation. Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.", - ) + expect(error.message).to.include("Undefined value encountered") ... - expect(error.message).to.equal( - "Undefined value encountered in property 'Post.text' of the find operation. Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.", - ) + expect(error.message).to.include("Undefined value encountered") ... - expect(error.message).to.equal( - "Null value encountered in property 'post.text' of the find operation. To match with SQL NULL, the IsNull() operator must be used. Set 'invalidWhereValuesBehavior.null' to 'ignore' or 'sql-null' in connection options to skip or handle null values.", - ) + expect(error.message).to.include("Null value encountered") ... - expect(error.message).to.equal( - "Null value encountered in property 'Post.text' of the find operation. To match with SQL NULL, the IsNull() operator must be used. Set 'invalidWhereValuesBehavior.null' to 'ignore' or 'sql-null' in connection options to skip or handle null values.", - ) + expect(error.message).to.include("Null value encountered") ... - expect(error.message).to.equal( - "Null value encountered in property 'Post.text' of the find operation. To match with SQL NULL, the IsNull() operator must be used. Set 'invalidWhereValuesBehavior.null' to 'ignore' or 'sql-null' in connection options to skip or handle null values.", - ) + expect(error.message).to.include("Null value encountered") ... - expect(error.message).to.equal( - "Undefined value encountered in property 'post.category.id' of the find operation. Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.", - ) + expect(error.message).to.include("Undefined value encountered") ... - expect(error.message).to.equal( - "Undefined value encountered in property 'Post.category' of the find operation. Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.", - ) + expect(error.message).to.include("Undefined value encountered") ... - expect(error.message).to.equal( - "Undefined value encountered in property 'Post.category' of the find operation. Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.", - ) + expect(error.message).to.include("Undefined value encountered")Also applies to: 369-371, 383-385, 641-643, 655-657, 671-673, 692-694, 708-709, 723-724
src/query-builder/SelectQueryBuilder.ts (4)
4308-4336: Undefined/null gating reads well; tiny perf nit.You recompute behaviors per key. Consider caching behaviors once per buildWhere call; low impact.
- if (parameterValue === undefined) { - const undefinedBehavior = - this.connection.options.invalidWhereValuesBehavior - ?.undefined || "ignore" + if (parameterValue === undefined) { + const undefinedBehavior = + this.connection.options.invalidWhereValuesBehavior?.undefined ?? "ignore"
4411-4426: Remove unreachable throw in relation null branch.When parameterValue === null, the earlier null-behavior block already throws/continues; only 'sql-null' falls through. The 'throw' here is dead code.
- if (where[key] === null) { + if (where[key] === null) { const nullBehavior = this.connection.options.invalidWhereValuesBehavior ?.null || "ignore" if (nullBehavior === "sql-null") { andConditions.push( `${alias}.${propertyPath} IS NULL`, ) - } else if (nullBehavior === "throw") { - throw new TypeORMError( - `Null value encountered in property '${alias}.${key}' of the find operation. ` + - `Set 'invalidWhereValuesBehavior.null' to 'ignore' or 'sql-null' in connection options to skip or handle null values.`, - ) } // 'ignore' behavior falls through to continue continue }
4430-4437: Undefined='throw' is bypassed for nested relation objects with all-undefined props.Current logic silently skips category: { id: undefined }. If undefined='throw', this should probably throw (mirroring top-level behavior).
- if (typeof where[key] === "object") { - const allAllUndefined = Object.keys(where[key]).every( - (k) => where[key][k] === undefined, - ) - if (allAllUndefined) { - continue - } - } + if (typeof where[key] === "object") { + const keys = Object.keys(where[key] as any) + const allUndef = keys.length > 0 && keys.every((k) => (where[key] as any)[k] === undefined) + if (allUndef) { + const undefinedBehavior = + this.connection.options.invalidWhereValuesBehavior?.undefined ?? "ignore" + if (undefinedBehavior === "throw") { + throw new TypeORMError( + `Undefined value encountered in property '${alias}.${propertyPath}.id' of the find operation. ` + + `Set 'invalidWhereValuesBehavior.undefined' to 'ignore' in connection options to skip properties with undefined values.`, + ) + } + continue + } + }
4355-4371: Minor ordering tweak: unwrap Equal() before null check.If Equal(null) is used, unwrapping first makes the subsequent null-path (IS NULL/behavior) explicit and consistent.
- if (parameterValue === null) { - andConditions.push(`${aliasPath} IS NULL`) - continue - } - - // todo: we need to handle other operators as well? - if (InstanceChecker.isEqualOperator(where[key])) { - parameterValue = where[key].value - } + // Unwrap Equal() first + if (InstanceChecker.isEqualOperator(where[key])) { + parameterValue = where[key].value + } + if (parameterValue === null) { + andConditions.push(`${aliasPath} IS NULL`) + continue + }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
docs/docs/data-source/2-data-source-options.md(1 hunks)docs/docs/data-source/5-null-and-undefined-handling.md(1 hunks)src/data-source/BaseDataSourceOptions.ts(1 hunks)src/query-builder/QueryBuilder.ts(1 hunks)src/query-builder/SelectQueryBuilder.ts(4 hunks)test/functional/find-options/empty-properties/find-options-empty-properties.ts(2 hunks)test/functional/null-undefined-handling/entity/Category.ts(1 hunks)test/functional/null-undefined-handling/entity/Post.ts(1 hunks)test/functional/null-undefined-handling/find-options.ts(1 hunks)test/functional/null-undefined-handling/query-builders.ts(1 hunks)test/utils/test-utils.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (6)
test/functional/null-undefined-handling/entity/Post.ts (4)
test/functional/null-undefined-handling/entity/Category.ts (1)
Entity(9-22)src/decorator/columns/PrimaryGeneratedColumn.ts (1)
PrimaryGeneratedColumn(55-119)src/decorator/relations/ManyToOne.ts (1)
ManyToOne(33-77)src/decorator/relations/JoinColumn.ts (1)
JoinColumn(31-48)
test/functional/null-undefined-handling/query-builders.ts (2)
test/utils/test-utils.ts (3)
createTestingConnections(388-482)reloadTestingDatabases(501-506)closeTestingConnections(487-496)src/error/TypeORMError.ts (1)
TypeORMError(1-17)
test/functional/null-undefined-handling/find-options.ts (2)
test/utils/test-utils.ts (3)
createTestingConnections(388-482)reloadTestingDatabases(501-506)closeTestingConnections(487-496)src/error/TypeORMError.ts (1)
TypeORMError(1-17)
test/functional/null-undefined-handling/entity/Category.ts (4)
test/functional/null-undefined-handling/entity/Post.ts (1)
Entity(10-24)src/decorator/columns/PrimaryGeneratedColumn.ts (1)
PrimaryGeneratedColumn(55-119)src/decorator/columns/Column.ts (1)
Column(134-220)src/decorator/relations/OneToMany.ts (1)
OneToMany(11-47)
src/query-builder/QueryBuilder.ts (1)
src/error/TypeORMError.ts (1)
TypeORMError(1-17)
src/query-builder/SelectQueryBuilder.ts (1)
src/error/TypeORMError.ts (1)
TypeORMError(1-17)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (20)
- GitHub Check: tests-linux (20) / postgres (14-3.5)
- GitHub Check: tests-linux (20) / cockroachdb
- GitHub Check: tests-linux (20) / postgres (17-3.5)
- GitHub Check: tests-linux (20) / sqljs
- GitHub Check: tests-linux (18) / oracle
- GitHub Check: tests-linux (20) / mongodb
- GitHub Check: tests-linux (18) / sap
- GitHub Check: tests-linux (18) / postgres (14-3.5)
- GitHub Check: tests-linux (18) / postgres (17-3.5)
- GitHub Check: tests-linux (18) / mysql_mariadb_latest
- GitHub Check: tests-linux (18) / sqljs
- GitHub Check: tests-linux (18) / better-sqlite3
- GitHub Check: tests-linux (18) / sqlite
- GitHub Check: tests-linux (18) / mysql_mariadb
- GitHub Check: tests-linux (18) / mssql
- GitHub Check: tests-linux (18) / mongodb
- GitHub Check: tests-windows / sqljs
- GitHub Check: tests-windows / better-sqlite3
- GitHub Check: tests-windows / sqlite
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (5)
test/functional/find-options/empty-properties/find-options-empty-properties.ts (1)
64-65: LGTM: clearer rationale for ts-expect-errorThe added comment clarifies that
nullis intentionally unsafe by default. Matches the documented guidance.Also applies to: 82-83
docs/docs/data-source/5-null-and-undefined-handling.md (1)
37-45: Link target verified — no change required.
Referenced file exists at docs/docs/working-with-entity-manager/3-find-options.md; the relative link ../working-with-entity-manager/3-find-options.md in docs/docs/data-source/5-null-and-undefined-handling.md is correct.test/functional/null-undefined-handling/entity/Category.ts (1)
9-22: LGTM: minimal, correct entity for tests.Mappings and nullable slug look good. No issues.
test/functional/null-undefined-handling/query-builders.ts (1)
46-63: LGTM: behavior coverage for qb/update/delete/softDelete.Assertions use .to.include for messages (good brittleness control) and validate .affected where appropriate.
Also applies to: 65-82, 84-101, 103-120, 122-139, 141-158
src/query-builder/SelectQueryBuilder.ts (1)
3163-3169: LGTM: relations-loading block insertion.Added relations/eager-relations handling after select reset is consistent with existing flow.
|
🎊 |
- added configurable handling for null/undefined in where clauses (ignore, SQL NULL, or throw) across queries, query builders, and repository/entity-manager methods
Summary by CodeRabbit
New Features
Documentation
Tests
Chores
Fixes #9316
Fixes #2500
Pull-Request Checklist
masterbranchnpm run formatto apply prettier formattingnpm run testpasses with this changeFixes #0000