Skip to content

claude_cli provider: concurrent temp-file collision on Windows drops most files from LLM analysis ([WinError 32]) #315

Closed

Description

@viniciusdeoliveiramed

Summary

On Windows, the claude_cli provider fails almost every LLM batch with [WinError 32] when a directory contains more than one file. The scan completes and reports 0/100, but only a small fraction of the components were actually analyzed.

v2.5.0's inspection ledger surfaces this correctly (thank you — that is exactly the right fix for the reporting side), so this issue is about the underlying race, not the reporting.

Environment

  • SkillSpector 2.5.0 (also reproduced on 2.4.4)
  • Windows 11 Pro 26200
  • SKILLSPECTOR_PROVIDER=claude_cli
  • Installed with uv tool install git+https://github.com/NVIDIA/skillspector.git

Steps to reproduce

  1. Clone any repo that has a directory with several code files, e.g.
    git clone --depth 1 https://github.com/DietrichGebert/ponytail
  2. set SKILLSPECTOR_PROVIDER=claude_cli
  3. skillspector scan ponytail/hooks --format markdown --output report.md

Actual result

stderr fills with one warning per file (20 warnings for 11 files, since it retries):

WARNING [skillspector.llm_analyzer_base] LLM batch failed for File: ponytail-activate.js:
  [WinError 32] The process cannot access the file because it is being used by another process:
  'C:\Users\<user>\AppData\Local\Temp\skillspector_cli_iuq6g_ey'

The report:

| Score          | 0/100   |
| Recommendation | CAUTION |

## Inspection Completeness
| Execution           | failed |
| Coverage            | 9.1%   |
| Fully inspected     | 1      |
| Partially inspected | 10     |

Exit code 2. Ten of eleven files land in Ledger Exceptions as llm_batch_failed.

Expected result

All components analyzed; Execution: successful, Coverage: 100.0%.

Isolation

The failure is concurrency-dependent, not file-dependent:

Target Files Exit Coverage
ponytail/skills/ponytail/ 1 0 100.0%
ponytail/hooks/ 11 2 9.1%

Scanning the same 11 files one at a time (each copied alone into its own temp directory) gives exit 0 and Coverage: 100.0% for all 11. Which files fail in a batch run varies between otherwise identical runs, which also points at a race rather than at any particular input.

Likely cause

skillspector_cli_* temp files appear to be created/opened/deleted concurrently by parallel batch workers. On Windows, an open handle blocks delete/rename (unlike POSIX), so a worker touching a temp path another worker still holds raises WinError 32.

Suggested fixes

  • Give each batch a uniquely-named temp file and delete it only after the handle is closed (tempfile.NamedTemporaryFile(delete=False) + explicit close() before os.unlink).
  • Or serialize the claude_cli provider's temp-file usage.
  • A --concurrency/--max-workers flag would also give users a workaround; there is currently no way to reduce parallelism from the CLI.

Why it matters

Before v2.5.0 this was silent: the same directory printed 0/100 / SAFE / exit 0 while only 1 of 11 files had been analyzed, indistinguishable from a complete clean scan. Anyone on Windows using claude_cli and reading only the score has been over-trusting their results. The v2.5.0 ledger makes it visible, but the scan is still effectively static-only for multi-file directories on Windows.

Activity

  1. pcaraujo6 commented on Sep 22, 2026

    @pcaraujo6

    Possible regression on v2.11.2 (main @ dabf4759), over a month after this was closed as completed. Different symptom text than the original [WinError 32], but same shape: claude_cli provider silently drops most files to static-only on Windows during a multi-file batch scan.

    Environment

    • SkillSpector 2.11.2 (git+https://github.com/NVIDIA/skillspector.git, installed via uv tool install)
    • Windows 10 Pro 10.0.19045
    • claude CLI 2.1.200 (Claude Code), authenticated, working session confirmed independently
    • SKILLSPECTOR_PROVIDER=claude_cli, no SKILLSPECTOR_MODEL override

    Command

    skillspector scan ~/.claude/plugins/cache/caveman --recursive --format json --output caveman-llm.json
    

    Target: a real, multi-file directory (171 components: markdown, JS, Python, shell, PowerShell).

    Observed

    stderr fills with one warning per affected file, e.g.:

    WARNING [skillspector.llm_analyzer_base] LLM structured response validation failed for File: caveman/0d95a81d35a9/docs/HONEST-NUMBERS.md after 4 attempts
    WARNING [skillspector.llm_analyzer_base] LLM structured response validation failed for File: caveman/0d95a81d35a9/plugins/caveman/skills/cavecrew/SKILL.md after 4 attempts
    WARNING [skillspector.llm_analyzer_base] LLM structured response validation failed for File: caveman/0d95a81d35a9/skills/caveman-compress/scripts/validate.py after 4 attempts
    [... 13 files total ...]
    WARNING [skillspector.nodes.report] LLM stage degraded: semantic runtime telemetry was incomplete; report may reflect static analysis only
    

    analysis_completeness in the JSON report:

    --no-llm run (same target) claude_cli run
    scanned_components 161 / 171 25 / 171
    coverage_percent 94.2% 14.6%
    fully_inspected_files 161 25
    partially_inspected_files 10 146
    status partial partial
    execution_successful true true

    Despite coverage collapsing from 94.2% to 14.6%, risk_assessment was identical in both runs:

    {"score": 100, "severity": "CRITICAL", "recommendation": "DO_NOT_INSTALL", "max_issue_severity": "HIGH"}

    Same dominant finding count too (RP1 × 142 in both). So the LLM pass didn't just fail to add value — it silently fell back to the static-only result while execution_successful: true and no non-zero exit code signaled the degradation. I only caught it because I diffed the two JSON reports side by side; anyone trusting a single claude_cli run and its score would have no reason to suspect 85% of the scan never ran through the LLM layer.

    Why I'm commenting here instead of opening a new issue

    This looks like the same underlying shape as the original report (multi-file directory, claude_cli provider, Windows, most files silently drop from LLM analysis, execution_successful still reports success) even though the concrete error text differs ([WinError 32] then vs. LLM structured response validation failed ... after 4 attempts now) — possibly a different manifestation of the same concurrency/temp-file race, or a second regression in the same code path. Flagging here for a maintainer to judge whether to reopen or track separately; happy to open a fresh issue instead if that's preferred.

    Full JSON reports (both runs) available on request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions