Skip to content

Comments

Mark indexed access object type comparisons as unreliable#52106

Merged
weswigham merged 1 commit intomicrosoft:mainfrom
weswigham:unreliable-indexed-accesses
Mar 1, 2023
Merged

Mark indexed access object type comparisons as unreliable#52106
weswigham merged 1 commit intomicrosoft:mainfrom
weswigham:unreliable-indexed-accesses

Conversation

@weswigham
Copy link
Member

@weswigham weswigham commented Jan 4, 2023

Fixes #52096

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jan 4, 2023
@weswigham
Copy link
Member Author

weswigham commented Jan 4, 2023

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023 •
edited
Loading

Heya @weswigham, I've started to run the perf test suite on this PR at 1864356. You can monitor the build here.

Update: The results are in!

@ahejlsberg
Copy link
Member

ahejlsberg commented Jan 4, 2023

@typescript-bot test this
@typescript-bot user test this inline
@typescript-bot run dt
@typescript-bot test top100

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023 •
edited
Loading

Heya @ahejlsberg, I've started to run the diff-based user code test suite on this PR at 1864356. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023 •
edited
Loading

Heya @ahejlsberg, I've started to run the diff-based top-repos suite on this PR at 1864356. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023 •
edited
Loading

Heya @ahejlsberg, I've started to run the extended test suite on this PR at 1864356. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023 •
edited
Loading

Heya @ahejlsberg, I've started to run the parallelized Definitely Typed test suite on this PR at 1864356. You can monitor the build here.

@ahejlsberg
Copy link
Member

ahejlsberg commented Jan 4, 2023

Anything that messes with variance-based vs. structural relations has me worried. Let's see what the test suites uncover.

@fatcerberus
Copy link

fatcerberus commented Jan 4, 2023

I’m confused - there don’t appear to be any indexed accesses in #52096?

@RyanCavanaugh
Copy link
Member

RyanCavanaugh commented Jan 4, 2023

Pick's definition uses indexed access types

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023

@ahejlsberg Here are the results of running the user test suite comparing main and refs/pull/52106/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023

Heya @ahejlsberg, I've run the RWC suite on this PR - assuming you're on the TS core team, you can view the resulting diff here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2023

@ahejlsberg Here are the results of running the top-repos suite comparing main and refs/pull/52106/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 5, 2023

@weswigham
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..52106
Metric main 52106 Delta Best Worst
Angular - node (v18.10.0, x64)
Memory used 372,498k (± 0.01%) 372,492k (± 0.01%) -7k (- 0.00%) 372,434k 372,532k
Parse Time 3.96s (± 0.81%) 4.00s (± 1.13%) +0.04s (+ 0.96%) 3.91s 4.04s
Bind Time 1.20s (± 0.73%) 1.20s (± 0.92%) +0.00s (+ 0.31%) 1.19s 1.22s
Check Time 8.62s (± 0.75%) 8.64s (± 0.40%) +0.02s (+ 0.21%) 8.58s 8.67s
Emit Time 7.55s (± 0.92%) 7.57s (± 1.09%) +0.02s (+ 0.32%) 7.49s 7.66s
Total Time 21.33s (± 0.59%) 21.41s (± 0.68%) +0.09s (+ 0.40%) 21.18s 21.55s
Compiler-Unions - node (v18.10.0, x64)
Memory used 199,110k (± 1.16%) 199,129k (± 1.17%) +19k (+ 0.01%) 198,139k 203,885k
Parse Time 1.65s (± 1.75%) 1.65s (± 1.30%) +0.00s (+ 0.07%) 1.62s 1.68s
Bind Time 0.79s (± 0.72%) 0.80s (± 1.26%) +0.01s (+ 1.02%) 0.79s 0.81s
Check Time 9.45s (± 0.66%) 9.46s (± 0.97%) +0.01s (+ 0.13%) 9.35s 9.60s
Emit Time 2.75s (± 1.04%) 2.76s (± 1.67%) +0.01s (+ 0.51%) 2.70s 2.81s
Total Time 14.64s (± 0.64%) 14.67s (± 0.99%) +0.04s (+ 0.25%) 14.48s 14.87s
Monaco - node (v18.10.0, x64)
Memory used 353,963k (± 0.01%) 353,967k (± 0.01%) +4k (+ 0.00%) 353,886k 354,018k
Parse Time 2.96s (± 1.34%) 2.94s (± 1.23%) -0.02s (- 0.57%) 2.89s 2.99s
Bind Time 1.04s (± 1.06%) 1.04s (± 0.49%) +0.00s (+ 0.09%) 1.04s 1.05s
Check Time 7.24s (± 0.66%) 7.22s (± 0.44%) -0.02s (- 0.34%) 7.17s 7.27s
Emit Time 4.31s (± 0.98%) 4.24s (± 0.90%) -0.07s (- 1.61%) 4.18s 4.30s
Total Time 15.55s (± 0.70%) 15.44s (± 0.55%) -0.11s (- 0.71%) 15.36s 15.58s
TFS - node (v18.10.0, x64)
Memory used 309,869k (± 0.01%) 309,865k (± 0.01%) -4k (- 0.00%) 309,823k 309,892k
Parse Time 2.25s (± 0.76%) 2.25s (± 1.39%) -0.00s (- 0.21%) 2.21s 2.30s
Bind Time 1.17s (± 0.88%) 1.16s (± 0.99%) -0.01s (- 0.78%) 1.14s 1.17s
Check Time 6.81s (± 0.75%) 6.80s (± 0.79%) -0.01s (- 0.21%) 6.71s 6.86s
Emit Time 3.94s (± 0.88%) 3.91s (± 1.13%) -0.03s (- 0.78%) 3.85s 3.96s
Total Time 14.17s (± 0.57%) 14.11s (± 0.73%) -0.06s (- 0.42%) 13.96s 14.23s
material-ui - node (v18.10.0, x64)
Memory used 485,044k (± 0.01%) 485,062k (± 0.01%) +17k (+ 0.00%) 485,030k 485,113k
Parse Time 3.52s (± 0.31%) 3.52s (± 0.45%) -0.00s (- 0.01%) 3.51s 3.54s
Bind Time 0.96s (± 0.55%) 0.97s (± 1.03%) +0.01s (+ 1.25%) 0.96s 0.99s
Check Time 16.96s (± 0.74%) 16.92s (± 0.71%) -0.04s (- 0.21%) 16.76s 17.12s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.44s (± 0.62%) 21.41s (± 0.56%) -0.03s (- 0.13%) 21.24s 21.60s
xstate - node (v18.10.0, x64)
Memory used 569,822k (± 0.02%) 570,459k (± 0.02%) +638k (+ 0.11%) 570,354k 570,621k
Parse Time 4.41s (± 0.87%) 4.44s (± 0.71%) +0.03s (+ 0.69%) 4.41s 4.49s
Bind Time 1.70s (± 0.74%) 1.70s (± 0.64%) -0.00s (- 0.10%) 1.68s 1.71s
Check Time 2.59s (± 0.75%) 2.61s (± 0.96%) +0.02s (+ 0.93%) 2.58s 2.65s
Emit Time 0.08s (± 0.11%) 0.08s (± 0.18%) -0.00s (- 0.10%) 0.08s 0.08s
Total Time 8.79s (± 0.47%) 8.84s (± 0.48%) +0.05s (+ 0.62%) 8.79s 8.91s
Angular - node (v16.17.1, x64)
Memory used 371,802k (± 0.01%) 371,847k (± 0.01%) +45k (+ 0.01%) 371,773k 371,896k
Parse Time 4.14s (± 0.68%) 4.14s (± 0.72%) -0.00s (- 0.04%) 4.11s 4.17s
Bind Time 1.27s (± 0.54%) 1.26s (± 0.48%) -0.01s (- 0.91%) 1.25s 1.27s
Check Time 9.28s (± 0.61%) 9.32s (± 0.50%) +0.04s (+ 0.41%) 9.27s 9.38s
Emit Time 7.96s (± 0.17%) 7.99s (± 0.65%) +0.02s (+ 0.29%) 7.92s 8.05s
Total Time 22.66s (± 0.34%) 22.71s (± 0.54%) +0.05s (+ 0.23%) 22.59s 22.84s
Compiler-Unions - node (v16.17.1, x64)
Memory used 200,839k (± 0.92%) 199,894k (± 0.03%) -945k (- 0.47%) 199,844k 199,959k
Parse Time 1.82s (± 0.51%) 1.82s (± 0.90%) -0.00s (- 0.17%) 1.80s 1.84s
Bind Time 0.84s (± 1.00%) 0.85s (± 1.19%) +0.00s (+ 0.33%) 0.83s 0.86s
Check Time 10.11s (± 0.83%) 10.19s (± 0.59%) +0.08s (+ 0.75%) 10.09s 10.26s
Emit Time 3.14s (± 6.00%) 2.97s (± 0.59%) 🟩-0.16s (- 5.24%) 2.95s 3.00s
Total Time 15.92s (± 1.43%) 15.82s (± 0.37%) -0.10s (- 0.60%) 15.72s 15.91s
Monaco - node (v16.17.1, x64)
Memory used 353,248k (± 0.01%) 353,243k (± 0.01%) -6k (- 0.00%) 353,200k 353,345k
Parse Time 3.19s (± 1.20%) 3.16s (± 0.62%) -0.02s (- 0.78%) 3.14s 3.19s
Bind Time 1.11s (± 1.22%) 1.12s (± 1.22%) +0.01s (+ 0.50%) 1.10s 1.14s
Check Time 7.85s (± 0.36%) 7.85s (± 0.49%) +0.00s (+ 0.01%) 7.81s 7.91s
Emit Time 4.48s (± 0.48%) 4.49s (± 0.47%) +0.01s (+ 0.16%) 4.45s 4.51s
Total Time 16.63s (± 0.36%) 16.62s (± 0.32%) -0.01s (- 0.07%) 16.55s 16.70s
TFS - node (v16.17.1, x64)
Memory used 309,212k (± 0.01%) 309,213k (± 0.00%) +1k (+ 0.00%) 309,199k 309,228k
Parse Time 2.62s (± 1.14%) 2.61s (± 1.10%) -0.01s (- 0.24%) 2.58s 2.65s
Bind Time 1.07s (± 1.43%) 1.05s (± 1.13%) -0.02s (- 1.74%) 1.04s 1.07s
Check Time 7.42s (± 0.49%) 7.36s (± 0.62%) -0.06s (- 0.81%) 7.31s 7.43s
Emit Time 4.24s (± 0.58%) 4.20s (± 1.24%) -0.04s (- 0.96%) 4.15s 4.27s
Total Time 15.34s (± 0.39%) 15.21s (± 0.71%) -0.13s (- 0.85%) 15.09s 15.38s
material-ui - node (v16.17.1, x64)
Memory used 484,374k (± 0.00%) 484,364k (± 0.00%) -10k (- 0.00%) 484,344k 484,379k
Parse Time 3.68s (± 0.39%) 3.67s (± 0.34%) -0.01s (- 0.26%) 3.66s 3.69s
Bind Time 1.02s (± 0.71%) 1.02s (± 0.64%) -0.00s (- 0.03%) 1.01s 1.03s
Check Time 17.86s (± 0.59%) 17.98s (± 1.35%) +0.12s (+ 0.68%) 17.82s 18.47s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 22.56s (± 0.45%) 22.67s (± 1.10%) +0.11s (+ 0.50%) 22.49s 23.17s
xstate - node (v16.17.1, x64)
Memory used 567,338k (± 0.02%) 567,858k (± 0.01%) +520k (+ 0.09%) 567,806k 568,001k
Parse Time 4.77s (± 0.73%) 4.75s (± 0.46%) -0.01s (- 0.31%) 4.73s 4.79s
Bind Time 1.66s (± 0.66%) 1.65s (± 0.68%) -0.00s (- 0.17%) 1.63s 1.66s
Check Time 2.83s (± 0.57%) 2.86s (± 0.62%) +0.03s (+ 0.99%) 2.83s 2.88s
Emit Time 0.09s (± 4.66%) 0.09s (± 0.15%) +0.00s (+ 1.82%) 0.09s 0.09s
Total Time 9.34s (± 0.37%) 9.35s (± 0.43%) +0.01s (+ 0.13%) 9.31s 9.42s
Angular - node (v14.15.1, x64)
Memory used 365,502k (± 0.00%) 365,502k (± 0.00%) +1k (+ 0.00%) 365,481k 365,514k
Parse Time 4.16s (± 0.79%) 4.17s (± 0.58%) +0.01s (+ 0.35%) 4.13s 4.20s
Bind Time 1.30s (± 0.49%) 1.31s (± 0.44%) +0.01s (+ 0.84%) 1.30s 1.32s
Check Time 9.66s (± 0.34%) 9.66s (± 0.53%) +0.00s (+ 0.05%) 9.60s 9.74s
Emit Time 8.21s (± 0.83%) 8.20s (± 0.67%) -0.00s (- 0.06%) 8.15s 8.29s
Total Time 23.32s (± 0.33%) 23.34s (± 0.39%) +0.02s (+ 0.09%) 23.25s 23.48s
Compiler-Unions - node (v14.15.1, x64)
Memory used 194,956k (± 0.01%) 194,912k (± 0.12%) -45k (- 0.02%) 194,445k 195,039k
Parse Time 1.86s (± 0.21%) 1.85s (± 0.40%) -0.01s (- 0.55%) 1.83s 1.85s
Bind Time 0.87s (± 0.56%) 0.88s (± 0.92%) +0.01s (+ 0.66%) 0.87s 0.90s
Check Time 10.14s (± 0.48%) 10.23s (± 2.02%) +0.09s (+ 0.90%) 10.11s 10.65s
Emit Time 3.12s (± 0.96%) 3.18s (± 3.65%) +0.06s (+ 1.87%) 3.12s 3.42s
Total Time 15.99s (± 0.38%) 16.14s (± 2.05%) +0.15s (+ 0.94%) 15.95s 16.81s
Monaco - node (v14.15.1, x64)
Memory used 348,122k (± 0.01%) 348,105k (± 0.01%) -17k (- 0.00%) 348,053k 348,159k
Parse Time 3.26s (± 1.01%) 3.24s (± 0.74%) -0.02s (- 0.53%) 3.20s 3.27s
Bind Time 1.13s (± 0.57%) 1.12s (± 0.39%) -0.00s (- 0.11%) 1.12s 1.13s
Check Time 8.13s (± 0.46%) 8.19s (± 0.35%) +0.06s (+ 0.75%) 8.16s 8.23s
Emit Time 4.70s (± 0.46%) 4.72s (± 0.58%) +0.02s (+ 0.39%) 4.68s 4.76s
Total Time 17.21s (± 0.41%) 17.27s (± 0.29%) +0.06s (+ 0.35%) 17.19s 17.31s
TFS - node (v14.15.1, x64)
Memory used 304,140k (± 0.00%) 304,142k (± 0.00%) +2k (+ 0.00%) 304,130k 304,152k
Parse Time 2.73s (± 0.83%) 2.74s (± 0.80%) +0.00s (+ 0.04%) 2.70s 2.76s
Bind Time 1.09s (± 0.60%) 1.08s (± 1.16%) -0.01s (- 0.50%) 1.07s 1.11s
Check Time 7.70s (± 0.38%) 7.70s (± 0.39%) +0.00s (+ 0.04%) 7.65s 7.74s
Emit Time 4.60s (± 1.00%) 4.57s (± 0.38%) -0.04s (- 0.85%) 4.55s 4.59s
Total Time 16.13s (± 0.45%) 16.09s (± 0.38%) -0.04s (- 0.25%) 15.97s 16.15s
material-ui - node (v14.15.1, x64)
Memory used 479,809k (± 0.00%) 479,792k (± 0.01%) -17k (- 0.00%) 479,730k 479,825k
Parse Time 3.91s (± 0.44%) 3.88s (± 0.53%) -0.02s (- 0.59%) 3.85s 3.91s
Bind Time 1.05s (± 0.68%) 1.05s (± 0.77%) -0.00s (- 0.03%) 1.04s 1.06s
Check Time 18.99s (± 0.56%) 19.08s (± 0.51%) +0.09s (+ 0.46%) 18.94s 19.19s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 23.95s (± 0.45%) 24.01s (± 0.46%) +0.07s (+ 0.28%) 23.85s 24.15s
xstate - node (v14.15.1, x64)
Memory used 555,710k (± 0.01%) 556,265k (± 0.01%) +555k (+ 0.10%) 556,224k 556,319k
Parse Time 5.12s (± 0.70%) 5.13s (± 0.55%) +0.00s (+ 0.10%) 5.10s 5.17s
Bind Time 1.60s (± 1.36%) 1.59s (± 0.88%) -0.00s (- 0.28%) 1.58s 1.61s
Check Time 2.99s (± 0.44%) 2.97s (± 0.61%) -0.02s (- 0.80%) 2.94s 2.99s
Emit Time 0.10s (± 0.16%) 0.10s (± 0.08%) +0.00s (+ 0.03%) 0.10s 0.10s
Total Time 9.81s (± 0.64%) 9.79s (± 0.41%) -0.03s (- 0.26%) 9.72s 9.82s
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 52106 6
Baseline main 6

TSServer

Comparison Report - main..52106
Metric main 52106 Delta Best Worst
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,537ms (± 1.73%) 2,558ms (± 1.64%) +21ms (+ 0.81%) 2,514ms 2,625ms
Req 2 - geterr 5,646ms (± 1.20%) 5,661ms (± 0.78%) +15ms (+ 0.26%) 5,606ms 5,734ms
Req 3 - references 362ms (± 2.69%) 364ms (± 0.80%) +2ms (+ 0.42%) 361ms 369ms
Req 4 - navto 291ms (± 1.50%) 292ms (± 2.72%) +2ms (+ 0.62%) 282ms 301ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 84ms (± 5.43%) 80ms (±10.19%) 🟩-5ms (- 5.63%) 69ms 89ms
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,696ms (± 1.05%) 2,740ms (± 1.61%) +43ms (+ 1.61%) 2,681ms 2,806ms
Req 2 - geterr 4,131ms (± 0.18%) 4,154ms (± 0.76%) +23ms (+ 0.54%) 4,109ms 4,189ms
Req 3 - references 357ms (± 0.76%) 358ms (± 0.75%) +0ms (+ 0.08%) 354ms 362ms
Req 4 - navto 290ms (± 0.74%) 294ms (± 0.66%) +4ms (+ 1.38%) 290ms 296ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 65ms (± 1.36%) 66ms (± 0.56%) +1ms (+ 2.06%) 66ms 67ms
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,625ms (± 0.71%) 3,596ms (± 0.52%) -29ms (- 0.80%) 3,565ms 3,615ms
Req 2 - geterr 1,331ms (± 1.32%) 1,351ms (± 1.45%) +20ms (+ 1.50%) 1,325ms 1,383ms
Req 3 - references 104ms (± 3.33%) 103ms (± 2.49%) -1ms (- 1.18%) 101ms 107ms
Req 4 - navto 360ms (± 1.93%) 354ms (± 1.28%) -5ms (- 1.49%) 350ms 362ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 435ms (± 1.46%) 431ms (± 1.53%) -4ms (- 0.84%) 424ms 442ms
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,792ms (± 0.59%) 2,769ms (± 1.08%) -22ms (- 0.80%) 2,728ms 2,812ms
Req 2 - geterr 6,035ms (± 0.81%) 6,060ms (± 0.61%) +25ms (+ 0.42%) 6,007ms 6,110ms
Req 3 - references 363ms (± 1.11%) 365ms (± 1.10%) +2ms (+ 0.62%) 361ms 369ms
Req 4 - navto 280ms (± 0.35%) 280ms (± 0.83%) -0ms (- 0.07%) 277ms 283ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 85ms (± 9.64%) 84ms (± 7.18%) -1ms (- 0.77%) 80ms 96ms
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,910ms (± 0.73%) 2,902ms (± 0.67%) -8ms (- 0.26%) 2,869ms 2,922ms
Req 2 - geterr 4,494ms (± 0.27%) 4,497ms (± 0.64%) +3ms (+ 0.07%) 4,472ms 4,537ms
Req 3 - references 383ms (± 1.06%) 385ms (± 0.56%) +2ms (+ 0.51%) 382ms 387ms
Req 4 - navto 296ms (± 1.09%) 296ms (± 1.04%) -0ms (- 0.07%) 291ms 300ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 74ms (± 3.97%) 73ms (± 3.47%) -1ms (- 1.28%) 71ms 77ms
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,762ms (± 0.30%) 3,777ms (± 0.59%) +15ms (+ 0.39%) 3,744ms 3,809ms
Req 2 - geterr 1,444ms (± 1.11%) 1,457ms (± 1.16%) +13ms (+ 0.88%) 1,429ms 1,480ms
Req 3 - references 108ms (± 1.00%) 113ms (± 2.56%) +5ms (+ 4.87%) 109ms 116ms
Req 4 - navto 352ms (± 0.70%) 346ms (± 0.75%) -6ms (- 1.76%) 343ms 350ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 455ms (± 1.93%) 462ms (± 1.56%) +6ms (+ 1.37%) 452ms 472ms
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,056ms (± 0.40%) 3,059ms (± 0.37%) +3ms (+ 0.08%) 3,045ms 3,072ms
Req 2 - geterr 5,907ms (± 0.66%) 5,912ms (± 0.51%) +5ms (+ 0.09%) 5,879ms 5,946ms
Req 3 - references 383ms (± 1.03%) 386ms (± 1.52%) +3ms (+ 0.91%) 379ms 393ms
Req 4 - navto 288ms (± 0.44%) 289ms (± 0.16%) +1ms (+ 0.20%) 288ms 290ms
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) 0 ( 0.00%) 1,356 1,356
Req 5 - completionInfo 82ms (± 1.20%) 82ms (± 0.56%) +0ms (+ 0.57%) 82ms 83ms
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,199ms (± 1.02%) 3,193ms (± 0.42%) -6ms (- 0.19%) 3,171ms 3,207ms
Req 2 - geterr 4,392ms (± 1.16%) 4,384ms (± 0.43%) -8ms (- 0.19%) 4,366ms 4,419ms
Req 3 - references 408ms (± 1.63%) 408ms (± 1.06%) -0ms (- 0.11%) 399ms 411ms
Req 4 - navto 308ms (± 2.47%) 307ms (± 1.86%) -1ms (- 0.32%) 300ms 313ms
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) 0 ( 0.00%) 1,518 1,518
Req 5 - completionInfo 82ms (± 7.39%) 85ms (± 9.39%) +3ms (+ 3.60%) 77ms 95ms
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,783ms (± 0.50%) 3,800ms (± 0.48%) +18ms (+ 0.47%) 3,784ms 3,829ms
Req 2 - geterr 1,460ms (± 0.95%) 1,478ms (± 0.60%) +18ms (+ 1.25%) 1,462ms 1,488ms
Req 3 - references 132ms (± 2.26%) 132ms (± 1.41%) +0ms (+ 0.10%) 129ms 134ms
Req 4 - navto 395ms (± 0.66%) 391ms (± 0.67%) -4ms (- 0.97%) 388ms 394ms
Req 5 - completionInfo count 3,154 (± 0.00%) 3,154 (± 0.00%) 0 ( 0.00%) 3,154 3,154
Req 5 - completionInfo 498ms (± 1.34%) 503ms (± 1.58%) +5ms (+ 0.99%) 494ms 516ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 52106 6
Baseline main 6

Startup

Comparison Report - main..52106
Metric main 52106 Delta Best Worst
tsc-startup - node (v16.17.1, x64)
Execution time 138.42ms (± 0.27%) 140.53ms (± 0.53%) +2.10ms (+ 1.52%) 137.80ms 144.61ms
tsserver-startup - node (v16.17.1, x64)
Execution time 222.62ms (± 0.19%) 225.03ms (± 0.49%) +2.41ms (+ 1.08%) 221.47ms 232.24ms
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 224.03ms (± 0.19%) 227.45ms (± 0.27%) +3.43ms (+ 1.53%) 223.36ms 230.89ms
typescript-startup - node (v16.17.1, x64)
Execution time 207.69ms (± 0.20%) 208.60ms (± 0.44%) +0.91ms (+ 0.44%) 205.49ms 216.37ms
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 52106 6
Baseline main 6

Developer Information:

Download Benchmark

@weswigham
Copy link
Member Author

weswigham commented Jan 5, 2023

All the suites and perf look fine, nice~

@RyanCavanaugh
Copy link
Member

RyanCavanaugh commented Jan 13, 2023

@ahejlsberg any remaining concerns?

@sandersn
Copy link
Member

sandersn commented Jan 19, 2023

@ahejlsberg @weswigham should this go in for 5.0 beta?

@sandersn
Copy link
Member

sandersn commented Feb 28, 2023

@weswigham I think we can get enough coverage by merging this at the beginning of 5.1, so I vote to merge it now.

@weswigham weswigham merged commit 958487b into microsoft:main Mar 1, 2023
@RyanCavanaugh RyanCavanaugh mentioned this pull request Apr 13, 2023
Closed
This was referenced Jun 5, 2023
Open
Closed
Closed
@DanielRosenwasser DanielRosenwasser mentioned this pull request Jun 26, 2023
Merged
@microsoft microsoft locked as resolved and limited conversation to collaborators Oct 22, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@ahejlsberg ahejlsberg Awaiting requested review from ahejlsberg

@RyanCavanaugh RyanCavanaugh Awaiting requested review from RyanCavanaugh

Assignees

@weswigham weswigham

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

PR Backlog
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uncomputed generic type does not follow structural typing

6 participants

@weswigham @typescript-bot @ahejlsberg @fatcerberus @RyanCavanaugh @sandersn