Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parallel builds of crate dependencies leads to intermittent linker errors #3161

Closed
tcosprojects opened this issue Oct 4, 2016 · 5 comments
Closed

Parallel builds of crate dependencies leads to intermittent linker errors #3161

tcosprojects opened this issue Oct 4, 2016 · 5 comments

Comments

@tcosprojects
Copy link

Summary

While cargo is compiling dependencies I'm seeing intermittent linker errors. It appears the issue is caused by how cargo uses job objects on windows detailed in this issue: rust-lang/rust#33145

Output

  Compiling log v0.3.6
 Documenting winapi v0.2.8
   Compiling rustc-serialize v0.3.19
   Compiling cfg-if v0.1.0
   Compiling utf8-ranges v0.1.3
   Compiling clang-sys v0.7.2
   Compiling strsim v0.5.1
   Compiling clang-sys v0.10.1
   Compiling winapi-build v0.1.1
   Compiling winapi v0.2.8
   Compiling backtrace-sys v0.1.4
   Compiling kernel32-sys v0.2.2
   Compiling dbghelp-sys v0.2.0
error: linking with `link.exe` failed: exit code: 1318
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LARGEADDRESSAWARE" "/SAFESEH" "/LIBPATH:<lots of paths to libs>
  = note: libpanic_unwind-40393716.rlib(panic_unwind-40393716.0.o) : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)'


error: aborting due to previous error

error: linking with `link.exe` failed: exit code: 1318
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LARGEADDRESSAWARE" "/SAFESEH" "/LIBPATH:<lots of paths to libs>
  = note: LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)'


error: aborting due to previous error

Build failed, waiting for other jobs to finish...
Build failed, waiting for other jobs to finish...

System info

Windows 10 x64

rustc 1.12.0 (3191fbae9 2016-09-23)
binary: rustc
commit-hash: 3191fbae9da539442351f883bdabcad0d72efcb6
commit-date: 2016-09-23
host: i686-pc-windows-msvc
release: 1.12.0

rustup 0.6.3 (a0e2132 2016-08-28)

cargo 0.13.0-nightly (109cb7c 2016-08-19)
@tcosprojects tcosprojects mentioned this issue Oct 4, 2016
Closed
@alexcrichton
Copy link
Member

Thanks for the report! You also mentioned that there are parallel builds of Cargo going in the background as well, which I think is a key ingredient here. In that sense, I think the sequence of steps here are:

  • Cargo process A starts
    • This process spawns rustc
    • This process spawns link.exe
    • This process spawns mspdbsrv.exe
  • Meanwhile, Cargo process B starts
    • This process spawns rustc, then link.exe
    • This link.exe realizes mspdbsrv.exe is already running, and starts communicating
  • Cargo process A finishes (link.exe returns, rustc.exe returns)
  • The job object for Cargo process A kills mspdbsrv.exe
  • Cargo process B's link.exe doesn't like mspdbsrv.exe going away, so it bails

I think the solution here is to do the same thing as the Rust build system which is to tweak our usage of job objects:

  • By default, do the same thing as today, tear down everything
  • If Cargo finishes entirely, then we kill all sub-processes that aren't whitelisted
  • We whitelist to persist mspdbsrv all the time

That way by default this scenario would work, and otherwise we'd still get the same benefits that if you interrupt things then everything is torn down.

@tcosprojects
Copy link
Author

To clarify, I'm using cargo interactively and launching cargo build once. I was under the impression that cargo built dependencies in parallel internally and that was where the conflict was coming from.

alexcrichton added a commit to alexcrichton/cargo that referenced this issue Oct 4, 2016
@alexcrichton
Leak mspdbsrv.exe processes on Windows
5ede71e 
Instead of having our job object tear them down, instead leak them intentionally
if everything succeeded.

Closes rust-lang#3161
@alexcrichton alexcrichton mentioned this issue Oct 4, 2016
Merged
bors added a commit that referenced this issue Oct 6, 2016
@bors
Auto merge of #3162 - alexcrichton:ugh-mspdbsrv, r=brson
505b9b0 
Leak mspdbsrv.exe processes on Windows

Instead of having our job object tear them down, instead leak them intentionally
if everything succeeded.

Closes #3161
@bors bors closed this as completed in #3162 Oct 6, 2016
@dimbleby dimbleby mentioned this issue Oct 23, 2017
Closed
@Geobert Geobert mentioned this issue Jan 7, 2018
Merged
@Nuxij
Copy link

Nuxij commented Mar 16, 2018

I'm using the script from this guy's blog to isolate mspdbsrv. http://blog.peter-b.co.uk/2017/02/stop-mspdbsrv-from-breaking-ci-build.html

  1. Run an mspdbsrv manually, overriding environment variable
  2. Run link.exe (or whatever else, devenv et al)
  3. Using python's 'with', mspdbsrv will automatically close, whether we succeed or fail.

@alexcrichton
Copy link
Member

@joeasaurus to confirm, is that required to fix this today? In theory this has been fixed in Cargo and shouldn't be an issue for Rust projects.

@Nuxij
Copy link

Nuxij commented May 4, 2018

@alexcrichton I don't think so, sorry for the necro post! Didn't look at the dates ;D

@Geobert Geobert mentioned this issue May 17, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexcrichton @tcosprojects @Nuxij