Skip to content
This repository was archived by the owner on Sep 1, 2026. It is now read-only.
This repository was archived by the owner on Sep 1, 2026. It is now read-only.

[bug] tsgo --build uses extreme amounts of memory #1622

Description

Stack trace

Unfortunately I don't have one to share as it crashed my remote dev machine (used 116GB of RSS before it did).

Image

Steps to reproduce

This is probably unique to our scale here at Canva. I attempted to build one of the TS projects in our monorepo that transitively depends upon 5k+ other TS projects. It built for ~5 mins before crashing out due to excess memory usage.

./built/local/tsgo --build ../canva/<omitted>/tsconfig.json

Activity

  1. sheetalkamat commented on Aug 22, 2025

    @sheetalkamat
    Member

    Can you please try #1623 and see if that helps

  2. mjames-c commented on Aug 22, 2025

    @mjames-c
    ContributorAuthor

    Sheetal Nandi (@sheetalkamat) I just tried it out again with your change included -- unfortunately still seems to be broken. memory climbed back up to 100GB RSS before the OS killed the process. we could try to take a pprof memory profile beginning of next week if you'd find it useful?

    Image
    $ strings ./built/local/tsgo | grep -i 15ae26ccc
    mod     github.com/microsoft/typescript-go      v0.0.0-20250822042943-15ae26cccfb7
    build   vcs.revision=15ae26cccfb7cd863aa009d0171715721d6785e1
  3. johnnycheng0210 commented on Aug 25, 2025

    @johnnycheng0210
    Contributor

    Sheetal Nandi (@sheetalkamat)

    following up Michael James (@mjames-c) -- with profiles attached for one of our largest TS projects:

    typescript-cpuprof.gz

    typescript-memprof.gz

    happy to provide some more profiles of smaller projects if that'll be useful as well.

  4. sheetalkamat commented on Aug 25, 2025

    @sheetalkamat
    Member

    Johnny Cheng (@johnnycheng0210) can you see if you repro this if you run --singleThreaded

  5. sheetalkamat commented on Sep 8, 2025

    @sheetalkamat
    Member

    Can you try #1692 to see if it helps. You can also use --maxConcurrentProjects to change how many projects can be built simultaneously and see if that is of any help

  6. hsource commented on Sep 11, 2025

    @hsource

    Sheetal Nandi (@sheetalkamat) We faced the same problem (even with --noEmit) of tsgo taking more memory than tsc. Using --singleThreaded resolved the problem, while still allowing us to take advantage of the performance gains. Thanks for the tip!

    Edit: One additional thing that can be tuned is GOGC (see docs) and GOMEMLIMIT (see docs). We set GOGC to 0.3 and GOMEMLIMIT to something less than our CI servers' limit

  7. sheetalkamat commented on Sep 22, 2025

    @sheetalkamat
    Member

    Can you please try out #1745 to see if it helps

    Thanks

  8. spanishpear commented on Sep 22, 2025

    @spanishpear

    Sheetal Nandi (@sheetalkamat) have been seeing this too on a mix of individual projects or large subsets (8k projects) in our internal monorepo, how can I help test the PR? Build locally?

    I might have missed where to find the branch deploy options 🙈🙏

  9. sheetalkamat commented on Sep 22, 2025

    @sheetalkamat
    Member

    You would need to build from #1745 to test it out

  10. 2 remaining items

  11. Knagis commented on Dec 9, 2025

    @Knagis

    7.0.0-dev.20251208.1

    I have the same issue in our repo, we have ~1000 ts projects, building everything in single process went up to 80GB commit size, based on error messages, didn't get much further than perhaps 5%, then crashed with oom.

    Running --singleThreaded works fine, everything compiles, result is still much much faster than with tsc.

  12. jakebailey commented on Dec 9, 2025

    @jakebailey
    Member

    Sheetal Nandi (@sheetalkamat) Can you resurrect your change to limit the number of concurrent project builds? We have --checkers, but I think we were also wanting to add --builders to control that number.

  13. jakebailey commented on Dec 10, 2025

    @jakebailey
    Member

    Tomorrow, can you all try build mode with --builders 1? Given single project checking is already concurrent, we think it might be that build more concurrency may not actually matter too much anymore.

  14. maxkostow commented on Dec 10, 2025

    @maxkostow

    Is that expected to impact the LSP or should there be a separate issue for the LSP?

  15. jakebailey commented on Dec 10, 2025

    @jakebailey
    Member

    No, that is largely unrelated and not the focus of this thread. Build mode is not a part of the language server.

    #2239 is closer, but I would very much suggest filing separate issues rather than commenting on existing ones if the cases aren't the same.

  16. Knagis commented on Dec 12, 2025

    @Knagis

    Jake Bailey (@jakebailey) I ran the test now with 7.0.0-dev.20251212.1

    --clean called before both runs.

    tsgo -b everything.json --singleThreaded - 3min 30sec to complete ~1000 tsconfig projects. ~30 sec to first error printed. Max commit size ~15GB

    tsgo -b everything.json --builders 1 - 5 minutes without any output, i manually killed it. max commit size ~30GB (reached early, didn't grow much afterwards). For long periods the process seems to be idle for noticeable time, then work again, then again idle (cpu and mem doesn't move).

  17. jakebailey commented on Dec 12, 2025

    @jakebailey
    Member

    Is this codebase public? That definitely sounds like the opposite of how it should behave.

  18. Knagis commented on Dec 12, 2025

    @Knagis

    Unfortunately, very private. Perhaps if you are willing, we could try NDA path to get access to it. Overall it is just a lot of react tsx files and jest and playwright spec files, with declaration emit only. Few symlinks between packages, but mostly uses path aliases still.

  19. Knagis commented on Dec 15, 2025

    @Knagis

    btw, tsgo -b everything.json --builders 1 --checkers 1 have the same behavior as simply --builders 1 - still 30GB commit size and hangs.

    Also the processes hang so that i can't even kill them via task manager - it shows access denied error.

    taskkill /f /im tsgo.exe
    ERROR: The process "tsgo.exe" with PID 14024 could not be terminated.
    Reason: There is no running instance of the task.
    ERROR: The process "tsgo.exe" with PID 36272 could not be terminated.
    Reason: There is no running instance of the task.
    
  20. spanishpear commented on Dec 30, 2025

    @spanishpear

    Jake Bailey (@jakebailey) Sheetal Nandi (@sheetalkamat) - Managed to see high memory by typechecking a larger project (and hence it's dependents). Interestingly, the tsc version has different aggregate stats, and is suspiciously "fast" 🤔 So either I've cooked the test (though I tried to run clean a lot!) or something else is afoot!

    When running on larger projects that went OOM, the pprof files were empty - so hopefully this is "large enough"? Let me know.
    Details and profiles below, let me know if I've missed anything (or misconfigured the runs).

    No rush on anything, just wanted to send while it was top of mind - happy holidays!

    $ tsc --version
    Version 5.9.2
    $ tsgo --version
    Version 7.0.0-dev.20251229.1
    

    tsgo

    tsgo --build ./foo/bar/baz/tsconfig.app.json --extendedDiagnostics --pprofDir $HOME/tmp/tsgo_profiling
    ExtendedDiagnostics
    Projects in scope:                       959
    Projects built:                          959
    Timestamps only updates:                   0
    Aggregate Files:                      802216
    Aggregate Lines:                   177931775
    Aggregate Identifiers:             170526142
    Aggregate Symbols:                 136494125
    Aggregate Types:                     8362450
    Aggregate Instantiations:           57375910
    Aggregate Memory used:          10166117116K
    Aggregate Memory allocs:        876822104759
    Aggregate Config time:               38.296s
    Aggregate BuildInfo read time:        0.017s
    Aggregate Parse time:               804.888s
    Aggregate Bind time:                  0.085s
    Aggregate Check time:               106.868s
    Aggregate Emit time:                546.251s
    Aggregate Changes compute time:      69.540s
    Aggregate Total time:               373.971s
    

    --builders=1

    tsgo --build ./foo/bar/baz/tsconfig.app.json--extendedDiagnostics --builders 1 --pprofDir $HOME/tmp/tsgo_profiling
    
    ExtendedDiagnostics
    Projects in scope:                       959
    Projects built:                          959
    Timestamps only updates:                   0
    Aggregate Files:                      802216
    Aggregate Lines:                   177931775
    Aggregate Identifiers:             170526142
    Aggregate Symbols:                 136494353
    Aggregate Types:                     8362430
    Aggregate Instantiations:           57375963
    Aggregate Memory used:           7369865774K
    Aggregate Memory allocs:        874861965624
    Aggregate Config time:               44.783s
    Aggregate BuildInfo read time:        0.008s
    Aggregate Parse time:                27.321s
    Aggregate Bind time:                  0.015s
    Aggregate Check time:                38.933s
    Aggregate Emit time:                138.541s
    Aggregate Changes compute time:       6.523s
    Aggregate Total time:              1120.233s
    

    tsc

    tsc --build ./foo/bar/baz/tsconfig.app.json --extendedDiagnostics
    ExtendedDiagnostics
    Projects in scope:                         959
    Projects built:                            959
    Aggregate Files:                        845214
    Aggregate Lines of Library:           48370444
    Aggregate Lines of Definitions:      131157035
    Aggregate Lines of TypeScript:         2980402
    Aggregate Lines of JavaScript:               0
    Aggregate Lines of JSON:                 15745
    Aggregate Lines of Other:                    0
    Aggregate Identifiers:               180215539
    Aggregate Symbols:                   129157294
    Aggregate Types:                       4518690
    Aggregate Instantiations:             21284442
    Aggregate Memory used:                3505480K
    Aggregate Assignability cache size:    2558219
    Aggregate Identity cache size:           88349
    Aggregate Subtype cache size:            60273
    Aggregate Strict subtype cache size:     85158
    Aggregate I/O Read time:                 8.60s
    Aggregate Parse time:                    5.31s
    Aggregate ResolveModule time:            8.11s
    Aggregate ResolveTypeReference time:     0.30s
    Aggregate ResolveLibrary time:           0.86s
    Aggregate Program time:                 49.96s
    Aggregate Bind time:                     2.63s
    Aggregate Check time:                   51.27s
    Aggregate transformTime time:           58.34s
    Aggregate commentTime time:              0.57s
    Aggregate I/O Write time:                4.45s
    Aggregate printTime time:               73.41s
    Aggregate Emit time:                    73.50s
    Config file parsing time:                3.95s
    Up-to-date check time:                   0.15s
    Build time:                            200.79s
    

    Profiles

    tsgo

    34030-cpuprofile.pb.gz
    34030-memprofile.pb.gz

    tsgo --builders=1

    54771-cpuprofile.pb.gz
    54771-memprofile.pb.gz

  21. spanishpear commented on Dec 30, 2025

    @spanishpear

    Sharing incase relevant- I also note that tsgolint has seen some similar performance struggles and there's some preliminary analysis pointing to resolution/strings in oxc-project/tsgolint#295 - though I'm not sure how much is tsgo, vs tsgolint's wrappers

  22. rubenferreira97 commented on Jan 7, 2026

    @rubenferreira97
  23. jakebailey commented on Jan 7, 2026

    @jakebailey
  24. sheetalkamat commented on Jan 29, 2026

    @sheetalkamat
    Member

    Can you please try build from #2604 to see if it helps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions