Skip to content

Comments

[red-knot] Also use alists for list of live bindings/declarations#16311

Closed
dcreager wants to merge 46 commits intomainfrom
dcreager/alist-for-live-bds
Closed

[red-knot] Also use alists for list of live bindings/declarations#16311
dcreager wants to merge 46 commits intomainfrom
dcreager/alist-for-live-bds

Conversation

@dcreager
Copy link
Member

This PR extends #16306 to also use association lists to store the list of live bindings and declarations for each symbol in the use-def map.

@dcreager dcreager added performance Potential performance improvement ty Multi-file analysis & type inference labels Feb 21, 2025
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much of the churn is this file is from the new parameters to thread through the SymbolStates arenas

@dcreager dcreager marked this pull request as ready for review February 21, 2025 22:45
@github-actions
Copy link
Contributor

github-actions bot commented Feb 21, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@carljm
Copy link
Contributor

carljm commented Feb 21, 2025

(Haven't reviewed this one yet, just wanted to note that CodSpeed seems to think this one is a 1% cold-check regression. So if that's consistent, we'll need to decide if there are sufficient code simplicity arguments to do it anyway, or if there's something we can do to eliminate the regression.)

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code changes look good to me (they seem mostly mechanical).

What would help me review this PR conceptually is a short explanation on how live bindings and declarations are used (what are the creation and access patterns) and why using alists is a good fit.


/// Restore the current builder symbols state to the given snapshot.
pub(super) fn restore(&mut self, snapshot: FlowSnapshot) {
pub(super) fn restore(&mut self, snapshot: &FlowSnapshot) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer accepting own instances for functions that unconditionally clone the parameter to make the cost more visible to callers. It can also help avoiding unnecessary calls in the cases where doesn't use the owned instance besides calling the function

.bindings
.iter(other.live_bindings)
.map(|(binding, _)| *binding);
if !self_bindings.eq(other_bindings) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to keep the len comparison to avoid comparing iterators of different lengths?

}
}

impl<I: Idx, K: Clone, V> ListBuilder<I, K, V> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'd probably move the Clone constraint closer to map to avoid that we accidentally "inherit" it for other methods adding to this ListBuilder impl block

Base automatically changed from dcreager/rework-bindings to main February 25, 2025 15:59
* main: (38 commits)
  [red-knot] Use arena-allocated association lists for narrowing constraints (#16306)
  [red-knot] Rewrite `Type::try_iterate()` to improve type inference and diagnostic messages (#16321)
  Add issue templates (#16213)
  Normalize inconsistent markdown headings in docstrings (#16364)
  [red-knot] Better diagnostics for method calls (#16362)
  [red-knot] Add argfile and windows glob path support (#16353)
  [red-knot] Handle pipe-errors gracefully (#16354)
  Rename `venv-path` to `python` (#16347)
  [red-knot] Fixup some formatting in `infer.rs` (#16348)
  [red-knot] Restrict visibility of more things in `class.rs` (#16346)
  [red-knot] Add diagnostic for class-object access to pure instance variables (#16036)
  Add `per-file-target-version` option (#16257)
  [PLW1507] Mark fix unsafe (#16343)
  [red-knot] Add a test to ensure that `KnownClass::try_from_file_and_name()` is kept up to date (#16326)
  Extract class and instance types (#16337)
  Re-order changelog entries for 0.9.7 (#16344)
  [red-knot] Add support for `@classmethod`s (#16305)
  Update Salsa (#16338)
  Update Salsa part 1 (#16340)
  Upgrade Rust toolchain to 1.85.0 (#16339)
  ...
* dcreager/dont-have-a-cow:
  Reuse unaliased cells in union
  Reuse unaliased cells in intersection
  Tracked aliased cells and update them in-place on insert
  Create structural property tests for union/intersection
  Add structural sharing property tests
  Don't clone any cells until we know we need to stitch up
  Add property tests
  Require explicit clones of alists
  Use named fields for list cells
* main:
  [red-knot] unify LoopState and saved_break_states (#16406)
  [`pylint`] Also reports `case np.nan`/`case math.nan` (`PLW0177`) (#16378)
  [FURB156] Do not consider docstring(s) (#16391)
  Use `is_none_or` in `stdlib-module-shadowing` (#16402)
  [red-knot] Upgrade salsa to include `AtomicPtr` perf improvement (#16398)
  [red-knot] Fix file watching for new non-project files (#16395)
  document MSRV policy (#16384)
  [red-knot] fix non-callable reporting for unions (#16387)
  bump MSRV to 1.83 (#16294)
  Avoid unnecessary info at non-trace server log level (#16389)
  Expand `ruff.configuration` to allow inline config (#16296)
  Start detecting version-related syntax errors in the parser (#16090)
* dcreager/dont-have-a-cow:
  [red-knot] unify LoopState and saved_break_states (#16406)
  [`pylint`] Also reports `case np.nan`/`case math.nan` (`PLW0177`) (#16378)
  [FURB156] Do not consider docstring(s) (#16391)
  Use `is_none_or` in `stdlib-module-shadowing` (#16402)
  [red-knot] Upgrade salsa to include `AtomicPtr` perf improvement (#16398)
  [red-knot] Fix file watching for new non-project files (#16395)
  document MSRV policy (#16384)
  [red-knot] fix non-callable reporting for unions (#16387)
  bump MSRV to 1.83 (#16294)
  Avoid unnecessary info at non-trace server log level (#16389)
  Expand `ruff.configuration` to allow inline config (#16296)
  Start detecting version-related syntax errors in the parser (#16090)
@codspeed-hq
Copy link

codspeed-hq bot commented Feb 27, 2025

CodSpeed Performance Report

Merging #16311 will degrade performances by 4.18%

Comparing dcreager/alist-for-live-bds (3fb1653) with main (7dad0c4)

Summary

❌ 1 regressions
✅ 31 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
red_knot_check_file[cold] 84.2 ms 87.8 ms -4.18%

dcreager added a commit that referenced this pull request Feb 28, 2025
Regardless of whether #16408 and #16311 pan out, this part is worth
pulling out as a separate PR.

Before, you had to define a new `IndexVec` index type for each type of
association list you wanted to create. Now there's a single index type
that's internal to the alist implementation, and you use `List<K, V>` to
store a handle to a particular list.

This also adds some property tests for the alist implementation.
@dcreager
Copy link
Member Author

I've tried a few variations on this, but have not been able to get a performance increase. I get things down to performance-neutral with the reusable alist cells from #16408, but that's not worth the extra code complexity.

@dcreager dcreager closed this Feb 28, 2025
@dcreager dcreager deleted the dcreager/alist-for-live-bds branch February 28, 2025 19:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Potential performance improvement ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants