node-gyp keeps failing unreliably on github windows ci. #2683
Description
-
Node Version: 12.x - 18.x
-
Platform: Windows (System Windows_NT 10.0.20348) Github Actions
-
Compiler: VS2022 (17.2.32602.215)
-
This issue happens in the Github CI with
windows-latestos, everything else is fine.
Setup is the matrix of windows with multiple nodejs version from v12 to v18. Because different version ofnpmare shipped with different versions of node, first step CI does is updatesnpmso project has latestnode-gypthat's shipped withnpm. Unfortunately, all of them fail, BUT even worse is they fail unreliably.. Sometimes test will pass, but most of the time it fails. So I am not sure if it's issue with node-gyp or not. But all of the failures happen when looking forcommon.gypifile.
Looking at the code, it seemscommon.gypifile is not being found atC:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\include\\node\\common.gypiand then it fallbacks to what is shown in the outputC:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\common.gypi.
Here are the example of it failing multiple times:- 2 succeed, but others fail: https://github.com/nodech/hsd/actions/runs/2515981257/attempts/2
- only 1 succeeds: https://github.com/nodech/hsd/actions/runs/2515981257
- 4 are successfull: https://github.com/nodech/hsd/actions/runs/2515981257/attempts/1
This happens inconsistently and same test will sometimes pass, mostly fail.
Interesting thing to note is that, the dependency itself is building fine: https://github.com/nodech/bcrypto/runs/6939480240, it only fails withhsdwhich has multiplenativedependencies other thanbcrypto. I tried to remove all other deps and only leave bcrypto. Switch from matrix to separate tests and etc. Always end up in the similar situation.
Example output for the failure from the Github CI:
npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
npm ERR! code 1
npm ERR! path D:\a\hsd\hsd\node_modules\bcrypto
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node-gyp rebuild
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info find Python using Python version 3.9.13 found at "C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/node-v16.15.1-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/node-v16.15.1-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/SHASUMS256.txt
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-x86/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-x64/node.lib
npm ERR! gyp http GET https://nodejs.org/download/release/v16.15.1/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/win-x64/node.lib
npm ERR! gyp http 404 https://nodejs.org/download/release/v16.15.1/win-arm64/node.lib
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.15.1/win-x86/node.lib
npm ERR! gyp info find VS using VS2022 (17.2.32602.215) found at:
npm ERR! gyp info find VS "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\hostedtoolcache\windows\Python\3.9.13\x64\python3.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args 'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\gyp\\gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'msvs',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args 'D:\\a\\hsd\\hsd\\node_modules\\bcrypto\\build\\config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args 'C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args 'C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1\\common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\16.15.1',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=C:\\\\Users\\\\runneradmin\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\16.15.1\\\\<(target_arch)\\\\node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=D:\\a\\hsd\\hsd\\node_modules\\bcrypto',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'D:\\a\\hsd\\hsd\\node_modules\\bcrypto\\build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp: C:\Users\runneradmin\AppData\Local\node-gyp\Cache\16.15.1\common.gypi not found (cwd: D:\a\hsd\hsd\node_modules\bcrypto) while reading includes of binding.gyp while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.onCpExit (C:\npm\prefix\node_modules\npm\node_modules\node-gyp\lib\configure.js:261:16)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:527:28)
npm ERR! gyp ERR! stack at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Windows_NT 10.0.20348
npm ERR! gyp ERR! command "C:\\hostedtoolcache\\windows\\node\\16.15.1\\x64\\node.exe" "C:\\npm\\prefix\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
npm ERR! gyp ERR! cwd D:\a\hsd\hsd\node_modules\bcrypto
npm ERR! gyp ERR! node -v v16.15.1
npm ERR! gyp ERR! node-gyp -v v9.0.0
npm ERR! gyp ERR! not ok
(...)