Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize comparing intersections with common members #50329

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Optimize comparing intersections with common members #50329

wants to merge 3 commits into from
+35 −0

Conversation

weswigham
Copy link
Member

@weswigham weswigham commented Aug 17, 2022 •
edited
Loading

Fixes #50290 - this brings the comparison count in this repo back down to pre-51b346d levels (there's still a few more, but not many), and compilation time is reduced commensurately.

Fundamentally, what happened is that the changes in 51b346d caused some union lengths to no longer match up thanks to the new simplification rules, and thus skip the union member correspondence optimization in #41903 and require exhaustive comparison. Since the unions in question are 252 elements or so long, there are a lot of those exhaustive comparisons (as I said in our internal chat - 252^3 is suspiciously close to the map size limit of 2^24!). By adding this optimization that strips the common members from intersections before comparing them (before normalization), we can prevent a lot of those corresponding (or almost-corresponding, in the case of the linked issue) unions resulting from intersection normalization from getting compared.

@weswigham
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot typescript-bot added the For Milestone Bug PRs that fix a bug with a specific milestone label Aug 17, 2022
@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 17, 2022 •
edited
Loading

Heya @weswigham, I've started to run the perf test suite on this PR at 08946cd. You can monitor the build here.

Update: The results are in!

@weswigham
Copy link
Member Author

(Note, this actually builds react-data-grid 1s faster (16s vs 17s) on my workstation than 4.7! So I'm optimistic performance will be pretty good!)

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..50329
Metric main 50329 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 360,779k (± 0.02%) 360,834k (± 0.02%) +55k (+ 0.02%) 360,712k 361,049k
Parse Time 2.10s (± 0.61%) 2.10s (± 1.01%) +0.00s (+ 0.05%) 2.07s 2.17s
Bind Time 0.89s (± 0.50%) 0.89s (± 0.58%) +0.00s (+ 0.11%) 0.88s 0.90s
Check Time 6.03s (± 0.63%) 6.07s (± 0.46%) +0.04s (+ 0.61%) 6.00s 6.12s
Emit Time 6.16s (± 0.54%) 6.18s (± 0.85%) +0.02s (+ 0.37%) 6.10s 6.30s
Total Time 15.17s (± 0.25%) 15.24s (± 0.47%) +0.06s (+ 0.41%) 15.09s 15.40s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,470k (± 0.03%) 205,516k (± 0.04%) +46k (+ 0.02%) 205,293k 205,687k
Parse Time 0.83s (± 0.89%) 0.83s (± 1.11%) +0.00s (+ 0.24%) 0.81s 0.85s
Bind Time 0.53s (± 1.25%) 0.53s (± 1.77%) 0.00s ( 0.00%) 0.51s 0.55s
Check Time 7.39s (± 0.49%) 7.45s (± 0.63%) +0.06s (+ 0.81%) 7.33s 7.54s
Emit Time 2.51s (± 0.72%) 2.52s (± 1.07%) +0.01s (+ 0.32%) 2.46s 2.58s
Total Time 11.26s (± 0.38%) 11.33s (± 0.58%) +0.07s (+ 0.62%) 11.18s 11.48s
Monaco - node (v10.16.3, x64)
Memory used 344,009k (± 0.02%) 344,051k (± 0.02%) +42k (+ 0.01%) 343,906k 344,227k
Parse Time 1.61s (± 0.50%) 1.61s (± 0.52%) +0.00s (+ 0.06%) 1.59s 1.63s
Bind Time 0.75s (± 0.79%) 0.76s (± 0.76%) +0.01s (+ 1.06%) 0.75s 0.78s
Check Time 6.00s (± 0.34%) 6.03s (± 0.46%) +0.04s (+ 0.63%) 5.96s 6.09s
Emit Time 3.26s (± 0.70%) 3.26s (± 0.73%) -0.00s (- 0.15%) 3.21s 3.30s
Total Time 11.62s (± 0.21%) 11.66s (± 0.35%) +0.04s (+ 0.32%) 11.54s 11.75s
TFS - node (v10.16.3, x64)
Memory used 305,281k (± 0.02%) 305,230k (± 0.02%) -52k (- 0.02%) 305,086k 305,335k
Parse Time 1.29s (± 0.52%) 1.30s (± 1.03%) +0.01s (+ 0.70%) 1.27s 1.33s
Bind Time 0.72s (± 0.31%) 0.72s (± 0.62%) +0.00s (+ 0.42%) 0.71s 0.73s
Check Time 5.43s (± 0.49%) 5.48s (± 0.58%) +0.05s (+ 0.87%) 5.43s 5.56s
Emit Time 3.43s (± 0.86%) 3.46s (± 0.99%) +0.03s (+ 0.87%) 3.38s 3.55s
Total Time 10.88s (± 0.41%) 10.96s (± 0.45%) +0.09s (+ 0.79%) 10.88s 11.09s
material-ui - node (v10.16.3, x64)
Memory used 471,656k (± 0.01%) 472,952k (± 0.01%) +1,296k (+ 0.27%) 472,770k 473,030k
Parse Time 1.83s (± 0.50%) 1.84s (± 0.79%) +0.01s (+ 0.44%) 1.81s 1.87s
Bind Time 0.68s (± 1.95%) 0.68s (± 1.79%) -0.00s (- 0.59%) 0.66s 0.70s
Check Time 14.48s (± 0.49%) 14.63s (± 0.44%) +0.16s (+ 1.08%) 14.51s 14.79s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.99s (± 0.47%) 17.15s (± 0.41%) +0.16s (+ 0.96%) 17.02s 17.32s
xstate - node (v10.16.3, x64)
Memory used 582,124k (± 1.26%) 578,870k (± 0.01%) -3,254k (- 0.56%) 578,705k 578,966k
Parse Time 2.59s (± 0.52%) 2.61s (± 0.44%) +0.01s (+ 0.50%) 2.57s 2.62s
Bind Time 1.05s (± 0.86%) 1.04s (± 0.79%) -0.00s (- 0.00%) 1.03s 1.06s
Check Time 1.57s (± 0.80%) 1.60s (± 0.55%) +0.03s (+ 1.98%) 1.58s 1.62s
Emit Time 0.07s (± 3.14%) 0.07s (± 4.66%) +0.00s (+ 2.82%) 0.07s 0.08s
Total Time 5.28s (± 0.38%) 5.31s (± 0.31%) +0.03s (+ 0.63%) 5.29s 5.36s
Angular - node (v12.1.0, x64)
Memory used 338,262k (± 0.02%) 338,336k (± 0.02%) +74k (+ 0.02%) 338,183k 338,539k
Parse Time 2.09s (± 0.41%) 2.10s (± 0.53%) +0.02s (+ 0.77%) 2.08s 2.13s
Bind Time 0.86s (± 0.72%) 0.85s (± 0.43%) -0.00s (- 0.47%) 0.85s 0.86s
Check Time 5.83s (± 0.45%) 5.83s (± 0.42%) -0.00s (- 0.05%) 5.77s 5.89s
Emit Time 6.39s (± 0.45%) 6.38s (± 0.85%) -0.02s (- 0.23%) 6.29s 6.50s
Total Time 15.17s (± 0.25%) 15.16s (± 0.43%) -0.00s (- 0.01%) 15.05s 15.28s
Compiler-Unions - node (v12.1.0, x64)
Memory used 193,135k (± 0.09%) 192,926k (± 0.32%) -208k (- 0.11%) 190,490k 193,335k
Parse Time 0.82s (± 0.92%) 0.82s (± 0.92%) 0.00s ( 0.00%) 0.81s 0.84s
Bind Time 0.55s (± 0.95%) 0.55s (± 1.01%) +0.00s (+ 0.55%) 0.54s 0.56s
Check Time 6.87s (± 0.51%) 6.94s (± 0.64%) +0.07s (+ 0.95%) 6.84s 7.05s
Emit Time 2.53s (± 0.96%) 2.57s (± 1.14%) +0.03s (+ 1.26%) 2.51s 2.65s
Total Time 10.78s (± 0.49%) 10.88s (± 0.59%) +0.10s (+ 0.94%) 10.77s 11.00s
Monaco - node (v12.1.0, x64)
Memory used 327,004k (± 0.02%) 326,899k (± 0.02%) -104k (- 0.03%) 326,770k 327,030k
Parse Time 1.58s (± 0.71%) 1.58s (± 0.95%) -0.00s (- 0.19%) 1.54s 1.61s
Bind Time 0.74s (± 1.05%) 0.74s (± 0.49%) +0.00s (+ 0.40%) 0.74s 0.75s
Check Time 5.81s (± 0.61%) 5.83s (± 0.32%) +0.03s (+ 0.45%) 5.78s 5.87s
Emit Time 3.29s (± 0.53%) 3.29s (± 0.41%) +0.00s (+ 0.06%) 3.26s 3.32s
Total Time 11.42s (± 0.40%) 11.45s (± 0.22%) +0.03s (+ 0.23%) 11.39s 11.53s
TFS - node (v12.1.0, x64)
Memory used 289,861k (± 0.01%) 289,828k (± 0.06%) -33k (- 0.01%) 289,201k 290,030k
Parse Time 1.29s (± 0.65%) 1.31s (± 0.71%) +0.02s (+ 1.32%) 1.29s 1.34s
Bind Time 0.71s (± 0.69%) 0.71s (± 0.73%) -0.00s (- 0.42%) 0.70s 0.72s
Check Time 5.38s (± 0.37%) 5.39s (± 0.47%) +0.01s (+ 0.19%) 5.33s 5.44s
Emit Time 3.50s (± 1.06%) 3.52s (± 0.74%) +0.02s (+ 0.54%) 3.48s 3.60s
Total Time 10.89s (± 0.55%) 10.93s (± 0.37%) +0.04s (+ 0.39%) 10.85s 11.04s
material-ui - node (v12.1.0, x64)
Memory used 450,762k (± 0.02%) 451,884k (± 0.08%) +1,122k (+ 0.25%) 450,948k 452,365k
Parse Time 1.82s (± 0.41%) 1.82s (± 0.44%) -0.00s (- 0.11%) 1.80s 1.83s
Bind Time 0.67s (± 0.55%) 0.67s (± 0.92%) +0.00s (+ 0.45%) 0.66s 0.69s
Check Time 13.06s (± 0.84%) 13.17s (± 1.04%) +0.11s (+ 0.86%) 12.94s 13.63s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.54s (± 0.71%) 15.66s (± 0.87%) +0.11s (+ 0.73%) 15.43s 16.12s
xstate - node (v12.1.0, x64)
Memory used 547,400k (± 1.31%) 544,248k (± 0.01%) -3,151k (- 0.58%) 544,101k 544,453k
Parse Time 2.52s (± 0.53%) 2.54s (± 0.68%) +0.01s (+ 0.48%) 2.51s 2.58s
Bind Time 1.00s (± 1.02%) 1.02s (± 0.94%) +0.01s (+ 1.50%) 0.99s 1.04s
Check Time 1.51s (± 0.82%) 1.52s (± 0.59%) +0.02s (+ 0.99%) 1.50s 1.54s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.10s (± 0.55%) 5.15s (± 0.49%) +0.05s (+ 0.88%) 5.09s 5.20s
Angular - node (v14.15.1, x64)
Memory used 336,463k (± 0.01%) 336,485k (± 0.01%) +21k (+ 0.01%) 336,419k 336,579k
Parse Time 2.07s (± 0.33%) 2.07s (± 0.42%) +0.00s (+ 0.10%) 2.05s 2.09s
Bind Time 0.89s (± 0.55%) 0.90s (± 0.62%) +0.00s (+ 0.45%) 0.89s 0.91s
Check Time 5.84s (± 0.39%) 5.87s (± 0.53%) +0.03s (+ 0.51%) 5.79s 5.93s
Emit Time 6.38s (± 0.78%) 6.37s (± 0.42%) -0.00s (- 0.08%) 6.33s 6.44s
Total Time 15.17s (± 0.36%) 15.20s (± 0.33%) +0.03s (+ 0.20%) 15.11s 15.32s
Compiler-Unions - node (v14.15.1, x64)
Memory used 191,784k (± 0.02%) 191,805k (± 0.01%) +21k (+ 0.01%) 191,771k 191,886k
Parse Time 0.85s (± 0.35%) 0.85s (± 0.89%) +0.01s (+ 0.83%) 0.84s 0.88s
Bind Time 0.58s (± 0.77%) 0.58s (± 0.57%) +0.00s (+ 0.35%) 0.57s 0.59s
Check Time 6.92s (± 0.78%) 6.94s (± 0.37%) +0.01s (+ 0.22%) 6.88s 6.99s
Emit Time 2.49s (± 1.14%) 2.49s (± 0.50%) -0.01s (- 0.36%) 2.45s 2.52s
Total Time 10.84s (± 0.59%) 10.86s (± 0.29%) +0.02s (+ 0.16%) 10.79s 10.94s
Monaco - node (v14.15.1, x64)
Memory used 325,769k (± 0.00%) 325,778k (± 0.01%) +9k (+ 0.00%) 325,730k 325,821k
Parse Time 1.57s (± 0.30%) 1.57s (± 0.43%) +0.01s (+ 0.45%) 1.56s 1.59s
Bind Time 0.78s (± 0.64%) 0.77s (± 0.64%) -0.00s (- 0.13%) 0.76s 0.78s
Check Time 5.68s (± 0.58%) 5.70s (± 0.44%) +0.02s (+ 0.26%) 5.63s 5.77s
Emit Time 3.36s (± 0.84%) 3.36s (± 0.84%) 0.00s ( 0.00%) 3.33s 3.44s
Total Time 11.38s (± 0.46%) 11.40s (± 0.34%) +0.02s (+ 0.18%) 11.30s 11.47s
TFS - node (v14.15.1, x64)
Memory used 288,884k (± 0.01%) 288,904k (± 0.00%) +20k (+ 0.01%) 288,875k 288,939k
Parse Time 1.35s (± 1.94%) 1.35s (± 2.14%) 0.00s ( 0.00%) 1.29s 1.42s
Bind Time 0.76s (± 3.63%) 0.77s (± 4.64%) +0.01s (+ 1.46%) 0.73s 0.87s
Check Time 5.37s (± 0.60%) 5.39s (± 0.33%) +0.01s (+ 0.28%) 5.34s 5.41s
Emit Time 3.65s (± 1.60%) 3.59s (± 2.13%) -0.06s (- 1.51%) 3.42s 3.74s
Total Time 11.13s (± 0.52%) 11.10s (± 0.78%) -0.03s (- 0.27%) 10.95s 11.25s
material-ui - node (v14.15.1, x64)
Memory used 448,821k (± 0.07%) 450,166k (± 0.06%) +1,345k (+ 0.30%) 449,094k 450,340k
Parse Time 1.88s (± 0.41%) 1.88s (± 0.56%) +0.00s (+ 0.27%) 1.85s 1.91s
Bind Time 0.72s (± 0.00%) 0.72s (± 1.01%) +0.00s (+ 0.28%) 0.70s 0.74s
Check Time 13.17s (± 0.86%) 13.26s (± 0.76%) +0.09s (+ 0.72%) 13.11s 13.51s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.76s (± 0.75%) 15.86s (± 0.61%) +0.10s (+ 0.65%) 15.72s 16.11s
xstate - node (v14.15.1, x64)
Memory used 542,068k (± 0.00%) 542,119k (± 0.00%) +50k (+ 0.01%) 542,086k 542,176k
Parse Time 2.59s (± 0.87%) 2.61s (± 0.41%) +0.01s (+ 0.54%) 2.58s 2.63s
Bind Time 1.13s (± 0.79%) 1.14s (± 1.49%) +0.01s (+ 1.07%) 1.10s 1.17s
Check Time 1.56s (± 0.56%) 1.58s (± 0.67%) +0.02s (+ 1.03%) 1.55s 1.60s
Emit Time 0.07s (± 4.66%) 0.07s (± 4.95%) +0.00s (+ 2.74%) 0.07s 0.08s
Total Time 5.35s (± 0.43%) 5.39s (± 0.68%) +0.05s (+ 0.86%) 5.32s 5.47s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50329 10
Baseline main 10

TSServer

Comparison Report - main..50329
Metric main 50329 Delta Best Worst
Compiler-UnionsTSServer - node (v10.16.3, x64)
Req 1 - updateOpen 1,455ms (± 0.58%) 1,454ms (± 0.48%) -1ms (- 0.08%) 1,434ms 1,463ms
Req 2 - geterr 3,795ms (± 0.52%) 3,805ms (± 0.52%) +10ms (+ 0.27%) 3,761ms 3,837ms
Req 3 - references 278ms (± 1.84%) 276ms (± 0.58%) -2ms (- 0.79%) 272ms 279ms
Req 4 - navto 231ms (± 1.46%) 232ms (± 1.37%) +1ms (+ 0.48%) 226ms 243ms
Req 5 - completionInfo count 1,355 (± 0.00%) 1,355 (± 0.00%) 0 ( 0.00%) 1,355 1,355
Req 5 - completionInfo 64ms (± 4.20%) 68ms (± 3.25%) +4ms (+ 5.75%) 62ms 71ms
CompilerTSServer - node (v10.16.3, x64)
Req 1 - updateOpen 1,560ms (± 0.71%) 1,563ms (± 0.45%) +3ms (+ 0.21%) 1,544ms 1,573ms
Req 2 - geterr 2,273ms (± 0.55%) 2,284ms (± 0.64%) +11ms (+ 0.50%) 2,251ms 2,306ms
Req 3 - references 296ms (± 0.83%) 301ms (± 1.35%) +6ms (+ 1.86%) 293ms 315ms
Req 4 - navto 236ms (± 2.00%) 235ms (± 2.67%) -1ms (- 0.34%) 222ms 244ms
Req 5 - completionInfo count 1,517 (± 0.00%) 1,517 (± 0.00%) 0 ( 0.00%) 1,517 1,517
Req 5 - completionInfo 67ms (± 2.46%) 65ms (± 2.16%) -2ms (- 2.83%) 63ms 70ms
xstateTSServer - node (v10.16.3, x64)
Req 1 - updateOpen 2,174ms (± 0.54%) 2,181ms (± 0.54%) +7ms (+ 0.33%) 2,153ms 2,199ms
Req 2 - geterr 781ms (± 0.65%) 782ms (± 0.51%) +1ms (+ 0.09%) 775ms 793ms
Req 3 - references 97ms (± 1.55%) 97ms (± 1.12%) +1ms (+ 0.73%) 95ms 100ms
Req 4 - navto 251ms (± 0.84%) 250ms (± 1.31%) -2ms (- 0.60%) 244ms 257ms
Req 5 - completionInfo count 3,244 (± 0.00%) 3,244 (± 0.00%) 0 ( 0.00%) 3,244 3,244
Req 5 - completionInfo 271ms (± 1.08%) 269ms (± 1.32%) -2ms (- 0.81%) 260ms 274ms
Compiler-UnionsTSServer - node (v12.1.0, x64)
Req 1 - updateOpen 1,483ms (± 0.52%) 1,488ms (± 0.45%) +5ms (+ 0.34%) 1,476ms 1,502ms
Req 2 - geterr 3,526ms (± 0.47%) 3,559ms (± 0.62%) +33ms (+ 0.94%) 3,500ms 3,599ms
Req 3 - references 263ms (± 0.80%) 264ms (± 0.85%) +0ms (+ 0.15%) 260ms 271ms
Req 4 - navto 204ms (± 0.85%) 205ms (± 1.08%) +2ms (+ 0.84%) 201ms 209ms
Req 5 - completionInfo count 1,355 (± 0.00%) 1,355 (± 0.00%) 0 ( 0.00%) 1,355 1,355
Req 5 - completionInfo 72ms (±14.67%) 83ms (±10.96%) +11ms (+15.90%) 63ms 97ms
CompilerTSServer - node (v12.1.0, x64)
Req 1 - updateOpen 1,574ms (± 0.60%) 1,567ms (± 0.45%) -7ms (- 0.44%) 1,553ms 1,585ms
Req 2 - geterr 2,230ms (± 0.67%) 2,225ms (± 0.63%) -5ms (- 0.24%) 2,189ms 2,254ms
Req 3 - references 273ms (± 1.17%) 274ms (± 0.63%) +1ms (+ 0.48%) 271ms 279ms
Req 4 - navto 216ms (± 1.05%) 216ms (± 1.17%) +1ms (+ 0.37%) 212ms 223ms
Req 5 - completionInfo count 1,517 (± 0.00%) 1,517 (± 0.00%) 0 ( 0.00%) 1,517 1,517
Req 5 - completionInfo 62ms (± 5.19%) 62ms (± 2.87%) +1ms (+ 0.97%) 58ms 66ms
xstateTSServer - node (v12.1.0, x64)
Req 1 - updateOpen 2,105ms (± 0.46%) 2,100ms (± 0.50%) -5ms (- 0.22%) 2,076ms 2,128ms
Req 2 - geterr 759ms (± 0.30%) 764ms (± 0.55%) +5ms (+ 0.66%) 755ms 774ms
Req 3 - references 65ms (± 1.56%) 66ms (± 0.93%) +2ms (+ 2.32%) 65ms 68ms
Req 4 - navto 239ms (± 1.28%) 240ms (± 0.73%) +1ms (+ 0.33%) 237ms 243ms
Req 5 - completionInfo count 3,244 (± 0.00%) 3,244 (± 0.00%) 0 ( 0.00%) 3,244 3,244
Req 5 - completionInfo 264ms (± 0.61%) 264ms (± 0.74%) -1ms (- 0.23%) 260ms 269ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,534ms (± 0.45%) 1,533ms (± 0.49%) -1ms (- 0.07%) 1,519ms 1,554ms
Req 2 - geterr 3,669ms (± 0.65%) 3,720ms (± 0.75%) +51ms (+ 1.40%) 3,676ms 3,798ms
Req 3 - references 275ms (± 0.49%) 276ms (± 0.81%) +1ms (+ 0.25%) 273ms 282ms
Req 4 - navto 219ms (± 0.70%) 219ms (± 0.87%) +0ms (+ 0.18%) 215ms 223ms
Req 5 - completionInfo count 1,355 (± 0.00%) 1,355 (± 0.00%) 0 ( 0.00%) 1,355 1,355
Req 5 - completionInfo 56ms (± 4.15%) 54ms (± 0.83%) 🟩-2ms (- 3.93%) 53ms 55ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 1,613ms (± 0.68%) 1,607ms (± 0.59%) -6ms (- 0.38%) 1,584ms 1,630ms
Req 2 - geterr 2,347ms (± 0.31%) 2,357ms (± 0.53%) +10ms (+ 0.42%) 2,339ms 2,395ms
Req 3 - references 287ms (± 1.13%) 285ms (± 0.59%) -2ms (- 0.73%) 282ms 290ms
Req 4 - navto 236ms (± 2.62%) 229ms (± 0.59%) 🟩-7ms (- 3.01%) 226ms 231ms
Req 5 - completionInfo count 1,517 (± 0.00%) 1,517 (± 0.00%) 0 ( 0.00%) 1,517 1,517
Req 5 - completionInfo 52ms (± 0.94%) 53ms (± 0.84%) +1ms (+ 1.53%) 52ms 54ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,220ms (± 0.61%) 2,214ms (± 0.27%) -6ms (- 0.27%) 2,199ms 2,224ms
Req 2 - geterr 775ms (± 0.22%) 782ms (± 0.59%) +7ms (+ 0.88%) 772ms 794ms
Req 3 - references 64ms (± 1.01%) 64ms (± 0.91%) -0ms (- 0.16%) 63ms 65ms
Req 4 - navto 247ms (± 0.45%) 248ms (± 0.63%) +1ms (+ 0.36%) 244ms 251ms
Req 5 - completionInfo count 3,244 (± 0.00%) 3,244 (± 0.00%) 0 ( 0.00%) 3,244 3,244
Req 5 - completionInfo 268ms (± 1.10%) 268ms (± 0.64%) +0ms (+ 0.07%) 264ms 271ms
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v10.16.3, x64)
  • Compiler-UnionsTSServer - node (v12.1.0, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v10.16.3, x64)
  • CompilerTSServer - node (v12.1.0, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v10.16.3, x64)
  • xstateTSServer - node (v12.1.0, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50329 10
Baseline main 10

Developer Information:

Download Benchmark

@ahejlsberg
Copy link
Member

@weswigham Would love to get the next level of detail on exactly what kinds of unions/intersections would match up before, but not now. Just trying to think deeper about the most efficient representation.

@weswigham
Copy link
Member Author

This PR essentially optimizes (A | B | C) & D -> (A' | B' | C') & D into just A | B | C -> A' | B' | C', which #41903 already ensures is then optimistically checked elementwise (A -> A', B -> B') to avoid exhaustive iteration in the success case. What seems to have happened in the linked issue, is that (A | B | C) & D -> (A' | B' | C') & D would sometimes normalize into A&D | B&D -> A'&C | B'&D | C'&D rather than A&D | B&D | C&D -> A'&C | B'&D | C'&D because C&D reduced to never for whatever reason (at least that's the reason I've come up with off hand, I didn't investigate the why too hard once I saw the normalized union size mismatch). Since A&D | B&D isn't a simple multiple of the length of A'&C | B'&D | C'&D, we'd miss out on the optimization from #41903 that frequently optimistically related unions in a linear number of comparisons rather than exponential, resulting in us exhaustively exploring the union members for relatability. Make those unions 252 elements long and the difference between the optimistic and exhaustive traversal of the unions becomes pretty large.

@weswigham
Copy link
Member Author

@typescript-bot pack this
@typescript-bot user test this inline
@typescript-bot test this
@typescript-bot run dt

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 17, 2022 •
edited
Loading

Heya @weswigham, I've started to run the tarball bundle task on this PR at 08946cd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 17, 2022 •
edited
Loading

Heya @weswigham, I've started to run the diff-based user code test suite on this PR at 08946cd. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 17, 2022 •
edited
Loading

Heya @weswigham, I've started to run the parallelized Definitely Typed test suite on this PR at 08946cd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 17, 2022 •
edited
Loading

Heya @weswigham, I've started to run the extended test suite on this PR at 08946cd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

@weswigham Here are the results of running the user test suite comparing main and refs/pull/50329/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Heya @weswigham, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 17, 2022 •
edited
Loading

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/132033/artifacts?artifactName=tgz&fileId=3E49C5820D2EEA586CB640CE3546A3038E966AF3FE23F4641D0FC615B8F2FDBF02&fileName=/typescript-4.9.0-insiders.20220817.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@ahejlsberg ahejlsberg mentioned this pull request Aug 20, 2022
Closed
@ehoogeveen-medweb ehoogeveen-medweb mentioned this pull request Aug 29, 2022
Closed
@gabritto gabritto mentioned this pull request Sep 25, 2023
Merged
@DanielRosenwasser
Copy link
Member

@typescript-bot perf test this
@typescript-bot test top300

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 28, 2023 •
edited
Loading

Heya @DanielRosenwasser, I've started to run the diff-based top-repos suite on this PR at bd83724. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 28, 2023 •
edited
Loading

Heya @DanielRosenwasser, I've started to run the regular perf test suite on this PR at bd83724. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 294,997k (± 0.01%) 294,989k (± 0.01%) ~ 294,965k 295,035k p=0.689 n=6
Parse Time 2.63s (± 0.57%) 2.63s (± 0.68%) ~ 2.60s 2.65s p=0.683 n=6
Bind Time 0.84s (± 0.97%) 0.84s (± 1.17%) ~ 0.83s 0.85s p=0.862 n=6
Check Time 8.07s (± 0.23%) 8.07s (± 0.40%) ~ 8.02s 8.11s p=0.808 n=6
Emit Time 7.05s (± 0.44%) 7.04s (± 0.25%) ~ 7.02s 7.07s p=0.744 n=6
Total Time 18.60s (± 0.23%) 18.59s (± 0.08%) ~ 18.56s 18.60s p=0.686 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 191,627k (± 1.23%) 193,870k (± 1.46%) ~ 190,545k 196,355k p=0.810 n=6
Parse Time 1.34s (± 0.41%) 1.36s (± 1.08%) ~ 1.34s 1.38s p=0.091 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.00%) ~ 0.73s 0.73s p=1.000 n=6
Check Time 9.20s (± 0.88%) 9.21s (± 0.43%) ~ 9.16s 9.27s p=1.000 n=6
Emit Time 2.63s (± 0.56%) 2.63s (± 0.34%) ~ 2.62s 2.64s p=1.000 n=6
Total Time 13.91s (± 0.60%) 13.92s (± 0.23%) ~ 13.89s 13.98s p=0.747 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,235k (± 0.00%) 347,253k (± 0.01%) ~ 347,214k 347,294k p=0.470 n=6
Parse Time 2.46s (± 0.31%) 2.46s (± 0.68%) ~ 2.44s 2.48s p=0.742 n=6
Bind Time 0.94s (± 0.00%) 0.94s (± 0.00%) ~ 0.94s 0.94s p=1.000 n=6
Check Time 6.89s (± 0.34%) 6.88s (± 0.34%) ~ 6.85s 6.91s p=0.627 n=6
Emit Time 4.01s (± 0.39%) 4.03s (± 0.75%) ~ 3.99s 4.08s p=0.328 n=6
Total Time 14.30s (± 0.22%) 14.31s (± 0.41%) ~ 14.22s 14.39s p=0.936 n=6
TFS - node (v18.15.0, x64)
Memory used 302,529k (± 0.01%) 302,521k (± 0.01%) ~ 302,505k 302,568k p=0.936 n=6
Parse Time 1.99s (± 0.78%) 2.01s (± 1.12%) ~ 1.98s 2.03s p=0.459 n=6
Bind Time 1.00s (± 0.63%) 1.00s (± 1.17%) ~ 0.99s 1.02s p=1.000 n=6
Check Time 6.25s (± 0.24%) 6.26s (± 0.52%) ~ 6.20s 6.29s p=0.466 n=6
Emit Time 3.53s (± 0.51%) 3.51s (± 0.84%) ~ 3.47s 3.56s p=0.195 n=6
Total Time 12.77s (± 0.25%) 12.78s (± 0.43%) ~ 12.72s 12.87s p=0.871 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,484k (± 0.00%) 472,378k (± 0.00%) +1,894k (+ 0.40%) 472,358k 472,406k p=0.005 n=6
Parse Time 2.58s (± 0.29%) 2.58s (± 0.57%) ~ 2.57s 2.61s p=0.604 n=6
Bind Time 0.99s (± 0.52%) 0.99s (± 0.64%) ~ 0.98s 1.00s p=0.386 n=6
Check Time 16.61s (± 0.34%) 16.73s (± 0.69%) ~ 16.61s 16.90s p=0.064 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.19s (± 0.28%) 20.30s (± 0.61%) ~ 20.17s 20.47s p=0.127 n=6
xstate - node (v18.15.0, x64)
Memory used 512,613k (± 0.01%) 512,575k (± 0.02%) ~ 512,471k 512,735k p=0.471 n=6
Parse Time 3.27s (± 0.23%) 3.27s (± 0.30%) ~ 3.26s 3.28s p=0.554 n=6
Bind Time 1.55s (± 0.82%) 1.55s (± 0.49%) ~ 1.54s 1.56s p=1.000 n=6
Check Time 2.84s (± 0.86%) 2.87s (± 0.93%) ~ 2.82s 2.89s p=0.063 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 6.73%) ~ 0.07s 0.08s p=0.174 n=6
Total Time 7.73s (± 0.25%) 7.75s (± 0.30%) ~ 7.71s 7.78s p=0.061 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,357ms (± 1.13%) 2,370ms (± 1.34%) ~ 2,331ms 2,412ms p=0.689 n=6
Req 2 - geterr 5,389ms (± 1.51%) 5,309ms (± 1.17%) ~ 5,241ms 5,414ms p=0.128 n=6
Req 3 - references 327ms (± 0.97%) 331ms (± 1.14%) ~ 326ms 335ms p=0.107 n=6
Req 4 - navto 276ms (± 1.27%) 279ms (± 1.37%) ~ 273ms 283ms p=0.224 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 85ms (± 6.69%) 78ms (± 7.68%) 🟩-8ms (- 9.00%) 74ms 89ms p=0.035 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,468ms (± 1.40%) 2,480ms (± 1.27%) ~ 2,421ms 2,500ms p=0.377 n=6
Req 2 - geterr 4,139ms (± 1.95%) 4,101ms (± 1.39%) ~ 4,065ms 4,216ms p=1.000 n=6
Req 3 - references 338ms (± 1.65%) 341ms (± 1.19%) ~ 333ms 344ms p=0.567 n=6
Req 4 - navto 284ms (± 0.43%) 284ms (± 0.48%) ~ 281ms 285ms p=0.933 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 81ms (± 8.43%) 85ms (± 6.25%) ~ 74ms 87ms p=0.655 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,598ms (± 0.60%) 2,598ms (± 0.25%) ~ 2,588ms 2,604ms p=0.261 n=6
Req 2 - geterr 1,671ms (± 2.10%) 1,695ms (± 2.02%) ~ 1,650ms 1,727ms p=0.336 n=6
Req 3 - references 109ms (± 7.69%) 116ms (± 9.76%) ~ 105ms 127ms p=0.324 n=6
Req 4 - navto 358ms (± 0.14%) 359ms (± 0.38%) ~ 357ms 361ms p=0.794 n=6
Req 5 - completionInfo count 2,071 (± 0.00%) 2,071 (± 0.00%) ~ 2,071 2,071 p=1.000 n=6
Req 5 - completionInfo 303ms (± 1.21%) 305ms (± 1.08%) ~ 299ms 308ms p=0.323 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 151.97ms (± 0.17%) 151.73ms (± 0.17%) -0.24ms (- 0.16%) 150.78ms 155.59ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 227.58ms (± 0.14%) 227.62ms (± 0.16%) ~ 226.26ms 232.34ms p=0.866 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 228.97ms (± 0.15%) 228.84ms (± 0.17%) -0.13ms (- 0.06%) 227.24ms 235.89ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 228.91ms (± 0.16%) 228.95ms (± 0.17%) ~ 227.62ms 234.52ms p=0.333 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser Here are the results of running the top-repos suite comparing main and refs/pull/50329/merge:

Everything looks good!

@ahejlsberg
Copy link
Member

ahejlsberg commented Sep 28, 2023 •
edited
Loading

@weswigham I wonder if we can do better here by examining the origin properties of unions resulting from intersections of other unions? For example, the unions resulting from intersections in (A | B | C) & D -> (A' | B' | C') & D are likely to have origin types that we could examine to determine that D can be removed from both. This would be a lot faster than examining every constituent of both of the normalized unions.

This would be somewhat similar to the optimizations in #55851.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ahejlsberg ahejlsberg Awaiting requested review from ahejlsberg

@RyanCavanaugh RyanCavanaugh Awaiting requested review from RyanCavanaugh

At least 1 approving review is required to merge this pull request.

Assignees

@weswigham weswigham

Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
PR Backlog
Status: Waiting on reviewers
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

RangeError: Map maximum size exceeded with [email protected]
5 participants
@weswigham @typescript-bot @ahejlsberg @DanielRosenwasser @RyanCavanaugh