Skip to content

Ingest layer (URL / zip / git clone) is unbounded upstream of the 50 MiB per-file gate #131

Closed

Description

@wernerkasselman-au

Summary

The per-file read cap added by the bounded-reads work (50 MiB, enforced in build_context._validate_file_sizes and the analyzer guards) sits downstream of InputHandler.resolve(), which pulls the scan target from a URL, zip, or git clone with no size budget of its own. A large download or a decompression bomb exhausts memory or disk before the per-file gate runs, so the 50 MiB guarantee does not hold for url/zip/git inputs.

Specifics (src/skillspector/input_handler.py)

  • _download_file (157-159): client.get(url) then response.content buffers the whole body in memory, no streaming, no Content-Length or byte-count limit. A multi-GB URL is a memory DoS.
  • _extract_zip (182): zf.extractall(extract_dir) with no uncompressed-size or file-count budget. A zip bomb fills the disk. (zipfile sanitises member names, so this is not Zip Slip / path traversal.)
  • _clone_git (131): git clone --depth 1 has a 60s timeout but no post-clone size cap; a large shallow repo still lands on disk.

Suggested direction

  • Stream URL downloads with a hard byte ceiling, abort once exceeded.
  • Bound zip extraction by total uncompressed size and member count (check ZipInfo.file_size before extracting).
  • Cap clone size (a disk-usage check after clone, or a partial-clone filter), and surface the cap.
  • Document in the README that 50 MiB is a per-file analysis limit, not an ingest limit.

Tests missing

zip-bomb, oversized-URL, oversized-clone, and single-file fail-closed coverage.

Surfaced during review of the bounded-reads PR (#19) by @rng1995.

Activity

  1. added a commit that references this issue on Jun 23, 2026
    cf31832
  2. added a commit that references this issue on Jul 14, 2026
    e5e5a21
  3. added a commit that references this issue on Jul 31, 2026
    e48fc1a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions