Skip to content

[ty] Ignore rejected assignments for synthesized bindings#25340

Merged
charliermarsh merged 8 commits into
mainfrom
charlie/reject-assignment
May 30, 2026
Merged

[ty] Ignore rejected assignments for synthesized bindings#25340
charliermarsh merged 8 commits into
mainfrom
charlie/reject-assignment

Conversation

@charliermarsh
Copy link
Copy Markdown
Member

@charliermarsh charliermarsh commented May 23, 2026

Summary

Prior to this change, we were retaining "known key" bindings from the right-hand side of an assignment even when the enclosing assignment was rejected. For example:

def _(x: dict[str, int]):
    x = {"a": "bad"}  # error: invalid-assignment
    reveal_type(x["a"])  # Should be int, but main shows Literal["bad"]

So we're respecting the new type information from x = {"a": "bad"}... But an invalid assignment shouldn't contribute new type information!

In other words, we want to ignore the known-key bindings that we get from the dict assignment when the overall assignment is invalid.

Doing so is pretty ugly because the index is built solely from syntax (e.g., we create definitions from x = {"a": "bad"}), but we're trying to reject those definitions later during inference, using information we don't have at the time of definition creation. I tried a variety of approaches but this ended up being the cheapest and least invasive thing I could find.

@astral-sh-bot astral-sh-bot Bot added the ty Multi-file analysis & type inference label May 23, 2026
@astral-sh-bot
Copy link
Copy Markdown

astral-sh-bot Bot commented May 23, 2026

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 91.94%. The percentage of expected errors that received a diagnostic held steady at 87.09%. The number of fully passing files held steady at 92/134.

@astral-sh-bot
Copy link
Copy Markdown

astral-sh-bot Bot commented May 23, 2026

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-argument-type 0 1 0
Total 0 1 0

Raw diff:

paasta (https://github.com/yelp/paasta)
- paasta_tools/frameworks/native_service_config.py:229:17 error[invalid-argument-type] Argument to bound method `list.append` is incorrect: Expected `dict[str, str | dict[str, int | float]]`, found `dict[str, str | dict[str, int | float] | dict[str, int | float | list[dict[str, int]]]]`

Full report with detailed diff (timing results)

@charliermarsh charliermarsh force-pushed the charlie/reject-assignment branch from f0ebbbe to 2f1767b Compare May 23, 2026 09:37
@astral-sh-bot
Copy link
Copy Markdown

astral-sh-bot Bot commented May 23, 2026

Memory usage report

Summary

Project Old New Diff Outcome
prefect 711.23MB 711.45MB +0.03% (228.42kB)
sphinx 261.68MB 261.77MB +0.03% (90.98kB)
trio 109.43MB 109.49MB +0.05% (60.15kB)
flake8 44.01MB 44.02MB +0.02% (9.54kB)

Significant changes

Click to expand detailed breakdown

prefect

Name Old New Diff Outcome
infer_definition_types 89.74MB 89.93MB +0.22% (200.23kB)
infer_deferred_types 10.47MB 10.49MB +0.11% (11.53kB)
infer_expression_types_impl 60.36MB 60.37MB +0.01% (7.04kB)
ClassType<'db>::abstract_methods_ 142.36kB 146.37kB +2.82% (4.01kB)
cached_protocol_interface 539.44kB 541.88kB +0.45% (2.44kB)
infer_scope_types_impl 50.85MB 50.85MB +0.00% (1.48kB)
place_by_id 4.90MB 4.90MB +0.01% (516.00B)
Type<'db>::member_lookup_with_policy_ 17.13MB 17.13MB +0.00% (420.00B)
enum_metadata 2.96MB 2.96MB +0.01% (216.00B)
infer_expression_type_impl 8.22MB 8.22MB +0.00% (108.00B)
StaticClassLiteral<'db>::try_metaclass_ 1.43MB 1.43MB +0.00% (72.00B)
StaticClassLiteral<'db>::inherited_legacy_generic_context_ 498.89kB 498.96kB +0.01% (72.00B)
StaticClassLiteral<'db>::explicit_bases_ 634.28kB 634.35kB +0.01% (72.00B)
StaticClassLiteral<'db>::decorators_ 518.38kB 518.45kB +0.01% (72.00B)
StaticClassLiteral<'db>::try_mro_ 5.34MB 5.34MB +0.00% (72.00B)
... 4 more

sphinx

Name Old New Diff Outcome
infer_definition_types 23.93MB 24.01MB +0.34% (83.57kB)
place_by_id 1.37MB 1.37MB +0.18% (2.50kB)
infer_expression_types_impl 21.96MB 21.96MB +0.01% (1.48kB)
infer_deferred_types 4.82MB 4.82MB +0.02% (1.07kB)
ClassType<'db>::abstract_methods_ 17.25kB 18.29kB +6.05% (1.04kB)
Type<'db>::member_lookup_with_policy_ 7.28MB 7.28MB +0.01% (840.00B)
infer_scope_types_impl 13.46MB 13.46MB +0.00% (276.00B)
cached_protocol_interface 273.66kB 273.79kB +0.05% (132.00B)
function_known_decorators 1.05MB 1.05MB +0.00% (48.00B)
overloads_and_implementation_inner 770.23kB 770.27kB +0.00% (36.00B)
enum_metadata 742.96kB 742.97kB +0.00% (12.00B)
StaticClassLiteral<'db>::decorators_ 122.00kB 122.02kB +0.01% (12.00B)

trio

Name Old New Diff Outcome
infer_definition_types 7.69MB 7.73MB +0.55% (43.66kB)
infer_expression_types_impl 6.75MB 6.75MB +0.09% (6.53kB)
place_by_id 546.89kB 549.53kB +0.48% (2.64kB)
infer_deferred_types 2.11MB 2.11MB +0.07% (1.57kB)
infer_expression_type_impl 1.36MB 1.36MB +0.11% (1.57kB)
ClassType<'db>::abstract_methods_ 32.14kB 33.42kB +3.97% (1.28kB)
infer_scope_types_impl 4.13MB 4.14MB +0.01% (636.00B)
function_known_decorators 297.47kB 298.02kB +0.19% (564.00B)
cached_protocol_interface 142.98kB 143.53kB +0.39% (564.00B)
Type<'db>::member_lookup_with_policy_ 1.93MB 1.93MB +0.01% (300.00B)
enum_metadata 258.43kB 258.72kB +0.11% (300.00B)
StaticClassLiteral<'db>::decorators_ 77.52kB 77.61kB +0.12% (96.00B)
all_narrowing_constraints_for_expression 612.44kB 612.51kB +0.01% (72.00B)
all_negative_narrowing_constraints_for_expression 591.54kB 591.61kB +0.01% (72.00B)
StaticClassLiteral<'db>::implicit_attribute_inner_ 738.73kB 738.79kB +0.01% (60.00B)
... 7 more

flake8

Name Old New Diff Outcome
infer_definition_types 1.84MB 1.85MB +0.46% (8.77kB)
ClassType<'db>::abstract_methods_ 2.14kB 2.32kB +8.20% (180.00B)
cached_protocol_interface 58.21kB 58.38kB +0.30% (180.00B)
infer_deferred_types 513.11kB 513.27kB +0.03% (172.00B)
enum_metadata 69.80kB 69.89kB +0.13% (96.00B)
infer_expression_types_impl 1.09MB 1.09MB +0.01% (60.00B)
Type<'db>::member_lookup_with_policy_ 573.86kB 573.89kB +0.01% (36.00B)
place_by_id 140.93kB 140.96kB +0.02% (36.00B)
function_known_decorators 156.08kB 156.11kB +0.02% (24.00B)

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 23, 2026

Merging this PR will improve performance by 7.69%

⚡ 1 improved benchmark
✅ 64 untouched benchmarks
⏩ 60 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime pydantic 37.1 s 34.5 s +7.69%

Tip

Curious why this is faster? Use the CodSpeed MCP and ask your agent.


Comparing charlie/reject-assignment (9984e4e) with main (042a6a1)

Open in CodSpeed

Footnotes

  1. 60 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@charliermarsh
Copy link
Copy Markdown
Member Author

Hate this diff.

@MichaReiser

This comment was marked as resolved.

@charliermarsh
Copy link
Copy Markdown
Member Author

I was hoping to snipe someone into it by posting about how much I didn't like the approach in Slack.

@charliermarsh charliermarsh force-pushed the charlie/non-never branch 2 times, most recently from d9f1196 to 3921ec4 Compare May 28, 2026 10:36
@charliermarsh charliermarsh marked this pull request as draft May 28, 2026 10:55
@charliermarsh charliermarsh force-pushed the charlie/reject-assignment branch from b01775e to 08d6118 Compare May 28, 2026 11:08
@charliermarsh charliermarsh marked this pull request as ready for review May 29, 2026 06:41
Copy link
Copy Markdown
Contributor

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took me a while to understand, but I think it makes sense to do it this way.

Thank you.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks completely unrelated to the changes in this PR... accidentally committed?

/// The fallback type for missing expressions/bindings/declarations or recursive type inference.
cycle_recovery: Option<Type<'db>>,

/// Whether refinements derived from the right-hand side of this definition were discarded.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"this definition" was a bit surprising to me given that a TypeInferenceBuilder could also be created for a non-definition region. Maybe it should be clarified here ("If the type inference region refers to a definition, ...")

}

fn discard_descendant_refinements_for(&mut self, definition: Definition<'db>) {
if matches!(self.region, InferenceRegion::Definition(region) if region == definition) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: region is not a region

Suggested change
if matches!(self.region, InferenceRegion::Definition(region) if region == definition) {
if matches!(self.region, InferenceRegion::Definition(d) if d == definition) {

cycle_recovery: Option<Type<'db>>,

/// Whether refinements derived from the right-hand side of this definition were discarded.
discards_descendant_refinements: bool,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that this flag says: this inference builder refers to a definition which resulted in an invalid-assignment error.

If that's true, I would prefer a less cryptic name like is_definition_with_invalid_assignment or invalid_assignment_definition?

Comment on lines +180 to +182
/// Such bindings describe refinement information derived from the right-hand side of their
/// enclosing assignment. If inference discards RHS-derived refinements for that assignment, its
/// synthesized descendants must not participate in place resolution either.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This goes over my head. I feel like a Python code example (from the PR description or a test) with a brief explanation would be more helpful than this prose description.

.finish_definition()
}

/// Returns `true` if a synthesized dictionary-key binding has been discarded.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Returns `true` if a synthesized dictionary-key binding has been discarded.
/// Returns `true` if the definition refers to a dictionary-key binding that resulted in an invalid-assignment.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar here: are these changes related to the PR?

Comment thread crates/ty_python_semantic/src/place.rs Outdated
Comment on lines +1407 to +1408
// The enclosing write already invalidated earlier child refinements. This
// rejected synthesized binding must not establish new RHS-derived precision.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. what? 😄

Doesn't this branch here mean that we skip a synthesized "x["a"] = 1" binding if the enclosing (real) x = {"a": 1} assignment (from which the synthesized was derived) was invalid?

Why does this refer to "invalidated earlier child refinements"? I understand that an assignment to x invalidates "earlier child refinements" like x["a"] = "foo", but how is that related to this branch?

Maybe

Suggested change
// The enclosing write already invalidated earlier child refinements. This
// rejected synthesized binding must not establish new RHS-derived precision.
// This synthesized binding was derived from a dictionary assignment
// `d = {...}` that was invalid. We discard it such that the lookup of the
// place `d[key]` can fail. A fallback place for `d[key]` can then be looked
// up on `d` directly.

@charliermarsh charliermarsh force-pushed the charlie/reject-assignment branch from e4ee3c2 to d12b7e2 Compare May 30, 2026 07:59
@charliermarsh charliermarsh changed the base branch from charlie/non-never to main May 30, 2026 08:23
@charliermarsh charliermarsh force-pushed the charlie/reject-assignment branch from d12b7e2 to 9984e4e Compare May 30, 2026 08:23
@charliermarsh charliermarsh merged commit 6c88390 into main May 30, 2026
58 of 59 checks passed
@charliermarsh charliermarsh deleted the charlie/reject-assignment branch May 30, 2026 09:05
carljm added a commit that referenced this pull request Jun 1, 2026
* main:
  [`pydocstyle`] Improve discoverability of rules enabled for each convention (#24973)
  [ty] Deduplicate retained use-def place states (#25450)
  [ty] reduce features of low-level crates depended on by `ty_python_semantic` (#25524)
  [ty] Fix narrowing enum literal unions by member identity (#25520)
  [ty] Test tagged union narrowing for named tuples (#25519)
  [ty] Disallow file-system access in `ty_python_core` (#25518)
  [ty] Nominal Tagged Union Narrowing (#24916)
  Commit `scripts/uv.lock` (#25517)
  Fix potential index out of range in `LineIndex` computation (#25492)
  [ty] Sync vendored typeshed stubs (#25514)
  [ty] Add disjointness for protocol method members (#25315)
  [ty] Use compact sets for more immutable fields (#25476)
  [ty] Derive `Default` for `FunctionDecoratorInference` (#25482)
  [ty] Ignore rejected assignments for synthesized bindings (#25340)
  [ty] Handle cycles in function decorator inference (#25475)
  docs: fix typo `bin/active` → `bin/activate` in tutorial (#25473)
  [ty] Narrow bound method overloads by receiver (#24707)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants