Skip to content

Commit 501b13e

Browse files
feat: RFC 3507 ICAP client with async, SSL/TLS, and pytest plugin (#20)
* added dockerfile for squidclamav docker image to test * added a comment * added a final test comment * fixed nested dict logic * cleaned up project files * Cleaned up request.py file * Added readme * Initial plan * Implement complete ICAP client and Docker integration testing setup Co-authored-by: CaptainDriftwood <[email protected]> * Add comprehensive documentation for Docker setup and implementation summary Co-authored-by: CaptainDriftwood <[email protected]> * Address code review feedback: improve error handling and response parsing Co-authored-by: CaptainDriftwood <[email protected]> * Refactor: extract EICAR constant and use uppercase hex formatting Co-authored-by: CaptainDriftwood <[email protected]> * Add logging, pytest support, pyproject.toml, and file scanning convenience methods Co-authored-by: CaptainDriftwood <[email protected]> * Fix numbering in README improvements list Co-authored-by: CaptainDriftwood <[email protected]> * Add type hints, GitHub workflows, pytest plugin, scan_bytes method, and integration tests Co-authored-by: CaptainDriftwood <[email protected]> * Fix code review issues: move imports to top level, add error handling, refactor scan methods Co-authored-by: CaptainDriftwood <[email protected]> * Add workflow permissions for security compliance Co-authored-by: CaptainDriftwood <[email protected]> * Add IcapServerError exception, Python 3.13 support, and use astral-sh/ruff-action Co-authored-by: CaptainDriftwood <[email protected]> * Pin ruff-action to stable version without explicit version param Co-authored-by: CaptainDriftwood <[email protected]> * feat(client): add exception handling, socket safety, and streaming support - Wire up custom exceptions (IcapConnectionError, IcapTimeoutError, IcapProtocolError, IcapServerError) throughout the client - Add null socket check in _send_and_receive to prevent AttributeError - Wrap socket operations with try/except to raise appropriate exceptions - Add chunked streaming support via scan_stream(chunk_size=N) parameter - Add _scan_stream_chunked() for memory-efficient large file scanning - Extract _receive_response() helper for response parsing * fix(pytest-plugin): add __all__ export for public API discoverability * chore: add TODO.md to gitignore * chore: add uv.lock for reproducible builds * feat: improve RFC 3507 compliance and add comprehensive documentation ICAP Protocol Improvements: - Implement chunked transfer encoding for RESPMOD body (RFC 3507 requirement) - Add chunked response parsing via _read_chunked_body() - Fix encapsulated header offset calculation for RESPMOD Testing Improvements: - Replace hardcoded 30s sleep with smart polling (wait_for_icap_service) - Add pytest-timeout for graceful test failure on hangs - Add separate integration test workflow with manual trigger - Migrate CI to use uv for dependency management Documentation: - Add comprehensive ICAP protocol explanation with architecture diagram - Document c-icap, SquidClamav, and ClamAV with official links - Update project structure (setup.py → pyproject.toml, add uv.lock) - Add Python 3.13 to supported versions * fix(docker): build squidclamav from source and fix c-icap configuration The c-icap-mod-virus-scan and squidclamav packages don't exist in Ubuntu 22.04 repositories. This commit fixes the Docker build by: - Building squidclamav v7.2 from source instead of using non-existent packages - Creating architecture-independent symlink for squidclamav.so module - Changing invalid LogLevel directive to DebugLevel in c-icap.conf - Adding ServicesDir directive to point to the correct module location - Making freshclam blocking to ensure virus definitions are loaded before clamd starts - Increasing retry attempts and adding better startup diagnostics * chore: add justfile with development recipes Add justfile with common development tasks: - install: Install dependencies with uv - check/ci: Run all checks (lint, typecheck, test) - test/test-integration/test-all: Run pytest with various filters - lint/lint-fix: Run ruff linter - fmt/fmt-check: Run ruff formatter - typecheck: Run pyright - docker-build/docker-up/docker-down/docker-logs: Docker operations - build: Build package - clean: Remove build artifacts * docs: improve README with error handling, EICAR examples, and tooling docs - Add note about PyPI name collision in installation section - Add error handling section with exception examples - Add EICAR test string example for virus detection testing - Fix docker compose commands to use correct path - Document just command runner as alternative to raw uv commands - Remove contributing section * refactor(docker): use multi-stage build to reduce image size by 65% - Implement multi-stage Dockerfile separating build and runtime stages - Remove build-essential and other compile-time dependencies from final image - Add ca-certificates and clamav-freshclam for proper SSL and virus updates - Use --no-install-recommends to minimize runtime dependencies - Remove obsolete 'version' key from docker-compose.yml - Increase healthcheck retries and add start_period for slower container startup - Add conftest.py to apply 300s timeout to integration tests Image size reduced from 602MB to 212MB. * chore: configure ruff isort and update fmt recipes - Add isort configuration with known-first-party for pycap - Update justfile fmt/fmt-check recipes to include import sorting - Apply ruff formatting fixes to icap.py * chore: filter testcontainers deprecation warning and document bug Filter the @wait_container_is_ready deprecation warning from testcontainers. The recommended replacement (HealthcheckWaitStrategy) has a bug with DockerCompose containers - it accesses wrapped.attrs which doesn't exist on ComposeContainer objects. Document the issue in test_integration.py with references to: - testcontainers/testcontainers-python#241 - testcontainers/testcontainers-python#144 Our ICAP-level polling approach is actually more robust as it verifies the protocol is responding, not just that the container is healthy. * docs: add table of contents and pytest plugin documentation Add navigable TOC with links to all sections. Document the pytest_pycap plugin fixtures (icap_client, icap_service_config, sample_clean_content, sample_file) with usage examples. * chore: remove empty request.py file Vestigial empty file that was never used. * docs: update project description to emphasize pure Python implementation * chore: remove obsolete documentation files and clean up README - Delete SUMMARY.md (historical PR summary no longer needed) - Delete IMPLEMENTATION_NOTES.md (content covered in README) - Remove "Key Improvements" section from README Future enhancements tracked in GitHub issues #2 and #3. * docs: move Docker setup guide to docker/README.md * feat: add AsyncIcapClient for async/await support - Add AsyncIcapClient with full async API matching sync client - Create shared _protocol.py module to reduce code duplication - Refactor IcapClient to use shared protocol base class - Add comprehensive async integration tests with pytest-asyncio - Move icap_service fixture to conftest.py for shared test usage - Switch type checker from pyright to ty - Add just ty/type aliases for type checking - Update README with async usage documentation * feat(pytest): add async_icap_client fixture to pytest plugin - Add async_icap_client fixture using async context manager pattern - Reuse existing @pytest.mark.icap marker for configuration - Update README with async fixture documentation and examples * feat(pytest): add Docker availability check to icap_service fixture Skip integration tests gracefully when Docker is not installed or the daemon is not running, providing clear error messages to help users understand why tests were skipped. * chore: add Python 3.14 support and consolidate dev dependencies - Add Python 3.14 to classifiers - Move dev dependencies from [project.optional-dependencies] to [dependency-groups] per PEP 735 (uv recommended approach) * chore: remove .claude from version control * feat: add public is_connected property to clients Add is_connected property to both IcapClient and AsyncIcapClient, replacing private attribute access throughout the codebase. Closes #6 * chore: add keywords to pyproject.toml for PyPI discoverability Closes #7 * test(pytest-plugin): add pytester tests for plugin validation Test the pytest_pycap plugin using pytester to verify: - @pytest.mark.icap marker registration - sample_clean_content fixture functionality - icap_service_config fixture structure - sample_file fixture creates temporary files - Plugin exports expected symbols - Client fixtures are importable Closes #5 * docs: add MIT LICENSE file Closes #4 * ci: add master branch to lint workflow triggers Closes #8 * ci: update typecheck workflow to use uv and ty - Switch from pip/pyright to uv/ty for consistency with local dev - Add master branch to workflow triggers Closes #9 * docs: fix Python version requirement in docker README (3.7 → 3.8) * refactor: use importlib.metadata for version (single source of truth) Version is now read from pyproject.toml via importlib.metadata, eliminating the need to maintain version in two places. * docs: improve ICAP protocol explanation and documentation accuracy - Rewrite "What is ICAP?" section for readers with basic HTTP knowledge - Add HTTP vs ICAP comparison table - Add section explaining Encapsulated header and chunked encoding - Add clickable link to RFC 3507 specification - Add Limitations section noting Preview mode is not yet implemented (#17) - Remove inaccurate "full RFC 3507" claim; list supported methods instead * feat: add ICAP preview mode support Add preview parameter to respmod method for both sync and async clients. Preview mode (RFC 3507) allows the server to make a decision based on the first N bytes without receiving the full payload, improving performance for large files. - Add preview parameter to IcapClient.respmod() and AsyncIcapClient.respmod() - Implement _send_with_preview() method for handling preview/continue flow - Support "ieof" chunk extension when entire body fits in preview - Handle 100 Continue response to send remainder of body - Add unit tests for preview functionality - Add integration tests for preview mode against docker fixture Closes #17 * feat: add SSL/TLS support for secure ICAP connections Add ssl_context parameter to both sync and async clients for establishing TLS-encrypted connections to ICAP servers. Supports standard TLS with system CA certificates, custom CA certificates, and client certificate authentication (mutual TLS). - Add ssl_context parameter to IcapClient and AsyncIcapClient - Wrap socket with SSL in sync client connect() method - Use asyncio.open_connection ssl parameter for async client - Handle ssl.SSLError and convert to IcapConnectionError - Add comprehensive docstrings with SSL usage examples - Add SSL/TLS section to README with usage examples - Add unit tests for SSL/TLS parameter handling - Remove Preview mode limitation from README (now implemented) Closes #16 * feat(pytest-plugin): add ssl_context support to fixtures Add ssl_context parameter support to icap_client and async_icap_client fixtures, allowing tests to configure TLS connections via the @pytest.mark.icap marker. - Add ssl_context to default config in both fixtures - Pass ssl_context to IcapClient and AsyncIcapClient constructors - Update docstrings with SSL/TLS usage examples - Update marker description to include ssl_context parameter - Add test for marker with ssl_context kwarg - Suppress pytest-asyncio deprecation warning from pytester subprocesses * chore: add SSL certificate generation infrastructure - Add `just generate-certs` command using Docker (no local openssl needed) - Create docker/certs/ directory with .gitignore for generated certs * chore(docker): add TLS support to ICAP server - Expose TLS port 11344 in Dockerfile and docker-compose.yml - Mount certs directory as volume for TLS certificates - Dynamically enable TLS in start.sh when certs are present - Update justfile generate-certs to include proper CA key usage extensions - Server works without certs (TLS disabled) for backwards compatibility * test: add SSL/TLS integration tests - Add icap_service_ssl fixture in conftest.py - Add test_ssl_integration.py with sync and async TLS tests - Add ssl marker to pyproject.toml - Update justfile: test-integration excludes ssl, test-integration-ssl includes all * fix: correct ieof placement in ICAP preview mode Per RFC 3507 Section 4.5, the "ieof" extension should be placed on the zero-length terminator chunk, not the data chunk. The previous implementation put ieof on the data chunk which caused c-icap to timeout waiting for additional data. Before: 13; ieof\r\n<data>\r\n0\r\n\r\n After: 13\r\n<data>\r\n0; ieof\r\n\r\n Fixed in both sync (IcapClient) and async (AsyncIcapClient) implementations. * docs(examples): add AWS Lambda handler for S3 virus scanning Add example Lambda function that demonstrates using PyCap to scan S3 objects for viruses on upload. The handler: - Uses aws-lambda-powertools for structured logging - Handles S3 CreateObject events - Downloads objects and scans with ICAP - Uses botocore ClientError for specific S3 error handling - Raises VirusFoundException when malware is detected * refactor(examples): improve lambda handler code quality - Remove redundant comments that explained "what" instead of "why" - Add virus name extraction from ICAP response headers - Include virus_name in VirusFoundException and error logs - Support common threat headers: X-Virus-ID, X-Infection-Found, etc. * fix: correct grammatical errors in docstrings - Add missing article "a" in port type error message - Add missing article "an" before "ICAP protocol error" - Add missing article "an" before "invalid response" in test - Add missing article "an" before "invalid port" in test * chore: remove PROJECT_ROADMAP.md from version control Move to .scratch/ directory for local use. Add .scratch/ to .gitignore for temporary working files. * perf: optimize Docker fixture for faster integration tests - Change icap_service fixture scope from module to session - Container now starts once per test session instead of per module - Add explicit image name to docker-compose.yml for better caching - Reduces integration test time by ~2/3 on subsequent runs * chore: clean up .gitignore by removing irrelevant templates Remove bloated Visual Studio/.NET, Windows Store, Azure, and other templates not relevant to this Python project. Consolidate patterns and remove blank lines and superfluous comments. * fix(ci): resolve Python 3.8 compatibility and lint workflow issues - Add `from __future__ import annotations` to tests/conftest.py for Python 3.8 compatibility with modern type hint syntax (tuple[], list[]) - Replace astral-sh/ruff-action with direct uv/ruff invocation in lint workflow to avoid transitive tj-actions/changed-files dependency (CVE-2025-30066 supply chain attack, also blocked by repo permissions) * ci: add CodeQL security scanning workflow Adds automated security scanning with GitHub CodeQL for Python code. Runs on PRs, pushes to main/master/develop, and weekly scheduled scans. Closes #11 * fix(ci): add actions:read permission to CodeQL workflow * feat: add nox for multi-version Python testing (3.8-3.14) - Add nox[uv] to dev dependencies for fast environment creation - Create noxfile.py with test, lint, and typecheck sessions - Add justfile recipes: test-all-versions, test-version, nox - Add .nox/ to gitignore Closes #19 * docs: add status badges to README - Python version badge (3.8 | 3.9 | 3.10 | 3.11 | 3.12 | 3.13 | 3.14) - MIT License badge - uv badge (Astral official) - Ruff badge (Astral official) Closes #10 * docs: rename PyCap to Pycap in README * docs: fix grammatical errors and incorrect tooling reference in README - "from the source" → "from source" - "See `docker/` directory" → "See the `docker/` directory" - "uv run pyright" → "uv run ty check pycap" (matches actual tooling) * docs: add ty badge to README * feat(pytest_pycap): add IcapResponseBuilder for fluent test responses Add a fluent builder pattern for creating IcapResponse objects in tests. Supports common scenarios like clean responses, virus detection, OPTIONS, errors, and 100 Continue. Example: response = IcapResponseBuilder().clean().build() response = IcapResponseBuilder().virus("Trojan.Generic").build() * feat(pytest_pycap): add MockIcapClient and MockAsyncIcapClient Add mock ICAP clients for testing without network I/O: - MockCall: dataclass for recording method calls - MockIcapClient: sync mock with full API, call recording, and assertions - MockAsyncIcapClient: async version with identical API Features: - Configurable responses via on_options(), on_respmod(), on_reqmod() - Exception injection for testing error handling - Call history with assert_called(), assert_not_called(), assert_scanned() - Full IcapClient interface (scan_bytes, scan_file, scan_stream, etc.) * feat(pytest_pycap): add mock fixtures for testing Add pytest fixtures for testing ICAP clients without network I/O: Response fixtures: - icap_response_builder: factory for custom responses - icap_response_clean: pre-built 204 No Modification - icap_response_virus: pre-built virus detection response - icap_response_options: pre-built OPTIONS response - icap_response_error: pre-built 500 error response Mock client fixtures: - mock_icap_client: sync mock with default clean responses - mock_async_icap_client: async mock with default clean responses - mock_icap_client_virus: configured to detect viruses - mock_icap_client_timeout: simulates timeouts - mock_icap_client_connection_error: simulates connection failures * feat(pytest_pycap): add icap_mock marker and fixture Add declarative configuration for mock clients via pytest marker: Usage: @pytest.mark.icap_mock(response="clean") def test_clean(icap_mock): ... @pytest.mark.icap_mock(response="virus", virus_name="Trojan.Gen") def test_virus(icap_mock): ... @pytest.mark.icap_mock(raises=IcapTimeoutError) def test_timeout(icap_mock): ... Supports per-method configuration: @pytest.mark.icap_mock( options={"response": "success"}, respmod={"response": "virus"}, ) * test(pytest_pycap): add comprehensive tests for mock components Add tests for all mock functionality: - IcapResponseBuilder (14 tests) - MockCall (1 test) - MockIcapClient (21 tests) - MockAsyncIcapClient (4 tests) - Mock fixtures (5 tests) - Response fixtures (5 tests) - icap_mock marker (8 tests) Also update plugin.py to export all new fixtures. * chore: register icap and icap_mock markers in pyproject.toml Add pytest marker definitions to prevent "unknown marker" warnings when using --strict-markers. * test(pytest_pycap): add comprehensive pytester tests for mock components Expand pytester test coverage from 9 to 27 tests: Mock fixture tests (6): - mock_icap_client, mock_async_icap_client - mock_icap_client_virus, mock_icap_client_timeout - mock_icap_client_connection_error Response fixture tests (4): - icap_response_builder, icap_response_clean - icap_response_virus, icap_response_error icap_mock marker tests (4): - Marker registration, virus response - Exception raising, per-method config Mock client usage tests (5): - Call recording, custom response configuration - Context manager, fluent builder API, exports * docs: expand pytest plugin documentation with mock API Add comprehensive documentation for the pytest plugin: - Live client fixtures (icap_client, async_icap_client) - Mock client fixtures (mock_icap_client, mock_async_icap_client, etc.) - Response fixtures (icap_response_clean, icap_response_virus, etc.) - IcapResponseBuilder fluent API with all methods - MockIcapClient configuration and assertion methods - @pytest.mark.icap_mock marker with all parameters Update table of contents to reflect new sections. * style: fix formatting in mock.py * ci(codeql): update codeql-action from v3 to v4 * ci: pin GitHub Actions to SHA hashes for supply chain security Pin all GitHub Actions to immutable commit SHAs instead of version tags to mitigate supply chain attacks like CVE-2025-30066 (tj-actions incident). Actions pinned: - actions/checkout@v4 - astral-sh/setup-uv@v5 - github/codeql-action/init@v4 - github/codeql-action/analyze@v4 * docs: enhance docstrings for LLM comprehension - Fix README.md pytest command and project structure - Add comprehensive docstrings to IcapResponse with attributes and examples - Document exception hierarchy with usage scenarios - Enhance IcapClient/AsyncIcapClient with API overview and thread safety - Improve pytest_pycap fixtures documentation with categories and guidance - Add detailed IcapResponseBuilder method documentation - Enhance MockIcapClient/MockAsyncIcapClient with configuration and assertion API * docs: refocus README overview on library value proposition Replace the redundant protocol method listing with highlights of the library's key features: sync/async clients, high-level scanning API, TLS support, pytest plugin, and zero dependencies. * style: add missing final newlines to tracked files * feat(pytest_pycap): add response sequence support to mock clients Add ability to queue multiple responses that are consumed in FIFO order. This enables testing scenarios where successive calls need different responses (e.g., first scan clean, second scan detects virus). Changes: - on_respmod(), on_options(), on_reqmod() now accept variadic *responses - Add _response_queues and _queue_active tracking to MockIcapClient - Add _get_method_response() for queue-aware response resolution - Add MockResponseExhaustedError when queue is depleted - Add reset_responses() to clear queues and restore defaults - Mirror all changes in MockAsyncIcapClient - Add 11 comprehensive tests for sequence behavior * feat(pytest_pycap): add stacked @pytest.mark.icap_response marker support Add declarative response sequences via stacked markers on test functions. Markers are processed in visual top-to-bottom order and support preset strings ("clean", "virus", "error") or IcapResponse objects. Changes: - Register icap_response marker in pytest_configure() - Add _resolve_marker_response() helper to convert markers to responses - Update icap_mock fixture to collect and apply stacked markers - Add 9 pytester-based integration tests for marker functionality Supports: - @pytest.mark.icap_response("clean") - @pytest.mark.icap_response("virus", virus_name="Trojan.Gen") - @pytest.mark.icap_response("error", code=503, message="Unavailable") - @pytest.mark.icap_response(response_object) * feat(pytest_pycap): add callback support for dynamic response generation Add callback= parameter to on_respmod() for dynamic response logic based on request content, filename, or service. Callbacks have highest priority in response resolution (before queues and defaults). Changes: - Add ResponseCallback and AsyncResponseCallback Protocol classes - Add _callbacks dict to MockIcapClient for per-method callbacks - Add callback= parameter to on_respmod() method - Update _get_method_response() to invoke callbacks first - Add _get_method_response_async() for async callback support - Support both sync and async callbacks in MockAsyncIcapClient - Export callback protocols from pytest_pycap package - Add 13 unit tests for callback behavior Example usage: def eicar_detector(data: bytes, **kwargs) -> IcapResponse: if b"EICAR" in data: return IcapResponseBuilder().virus("EICAR-Test").build() return IcapResponseBuilder().clean().build() client.on_respmod(callback=eicar_detector) * feat(pytest_pycap): add content matchers for declarative conditional responses Add when().respond() API for configuring conditional mock responses based on service name, filename, or content patterns. Matchers are checked first in response resolution, taking priority over callbacks, queues, and defaults. Features: - ResponseMatcher dataclass for storing match criteria - MatcherBuilder class for fluent API via when() - Support for exact filename, regex patterns, and data_contains - Combined criteria with AND logic - times= parameter for limited-use matchers - Method chaining for registering multiple matchers - Async client support via _get_method_response_async() Example: client.when(filename="malware.exe").respond(virus_response) client.when(data_contains=b"EICAR").respond(virus_response, times=2) * feat(pytest_pycap): add enhanced call history for rich inspection and assertions - Enhance MockCall dataclass with response, exception, matched_by, call_index fields - Add convenience properties: data, filename, service, succeeded, was_clean, was_virus - Improve MockCall.__repr__ for rich debugging output - Add call query methods: first_call, last_call, last_scan_call, get_calls, get_scan_calls - Add call statistics: call_count, call_counts_by_method - Add enhanced assertions: assert_called_with, assert_any_call, assert_called_in_order, assert_scanned_file, assert_scanned_with_filename - Mirror all changes in MockAsyncIcapClient - Add 56 unit tests for Phase 4 functionality * feat(pytest_pycap): add strict mode for unused response validation Add strict mode to MockIcapClient that validates all configured responses were consumed during tests. This helps catch test setup that doesn't match actual test behavior. - Add strict parameter to MockIcapClient.__init__ - Track queue sizes, callback usage, and matcher triggers - Add assert_all_responses_used() method for explicit validation - Add @pytest.mark.icap_mock(strict=True) marker support - Automatic teardown validation when strict mode enabled via marker - Add 20 new tests for strict mode functionality * docs: update MockIcapClient and README documentation - Update MockIcapClient class docstring with comprehensive overview of all 6 capabilities (response config, sequences, callbacks, matchers, call recording, strict mode) - Update mock.py module docstring with key features and examples - Expand README MockIcapClient section with examples for all features - Add stacked response markers documentation to README - Add strict mode parameter to icap_mock marker docs * fix: address multiple bugs in ICAP client error handling - Fix socket resource leak on connection error (BUG-001) - Close socket in exception handlers before setting to None - Fix async writer not properly closed on error (BUG-002) - Call writer.close() before setting to None in error handlers - Fix silent data truncation on incomplete response (BUG-003) - Validate bytes_read == content_length, raise IcapProtocolError if mismatch - Fix silent truncation on malformed chunk size (BUG-004) - Raise IcapProtocolError instead of returning partial data - Fix connection state not updated on socket errors (BUG-005) - Set _connected = False when socket errors occur in _receive_response() - Fix unhandled ValueError in Content-Length parsing (BUG-006) - Wrap int() in try/except, raise IcapProtocolError with context - Add preview size validation (BUG-007) - Validate preview parameter is positive integer * test: add comprehensive error handling tests Add 29 unit tests covering error handling and edge cases: - IcapProtocolError for malformed responses, invalid Content-Length, invalid chunk sizes - IcapServerError for 5xx responses (500, 502, 503, 505) - _read_chunked_body() with various inputs and edge cases - Connection lifecycle (reconnect, idempotent connect, safe disconnect) - scan_file() with non-existent file (FileNotFoundError) - Preview size validation (ValueError for zero/negative values) - Response parsing edge cases (empty body, Content-Length: 0, multi-word status) * fix: add proper exception chaining for lint compliance Add `from None` to re-raised exceptions in except clauses to satisfy ruff B904 rule and properly distinguish exceptions from error handling. * fix: address remaining code review issues - Fix silent chunked body truncation: raise IcapProtocolError when connection closes before chunked body complete (HIGH severity) - Wrap stream.read() errors: catch IOError/OSError and re-raise as IcapProtocolError with context (MEDIUM severity) - Add chunk_size parameter to AsyncIcapClient.scan_stream() for API parity with sync client (LOW severity) - Add 5 new tests covering these fixes * test: add comprehensive unit tests for client methods Add 36 new tests covering: - _send_with_preview() tests (timeout, connection error, 100 Continue flow) - _receive_response() tests (sync and async) - _read_chunked_body() tests (sync and async) - _scan_stream_chunked() tests - _iter_chunks() tests (sync and async) - reqmod() and options() public API tests - Real socket timeout and connection error scenarios - Sync/async API parity verification tests All tests pass (287 total, 26 deselected for integration/SSL). * refactor: migrate tests from unittest.mock.patch to pytest-mock - Add pytest-mock dependency for cleaner test mocking - Convert 6 tests from patch/patch.object to mocker.patch: - test_error_handling.py: 3 socket patching tests - test_client_methods.py: 3 auto-connect tests - Remove unused patch imports pytest-mock provides: - Automatic cleanup (no context managers needed) - Single consistent API via mocker fixture - Better integration with pytest ecosystem * refactor: mock network calls in connection error tests - Add 'slow' marker definition to pyproject.toml for future use - Mock socket connections in test_client_methods.py to avoid real network calls that cause slow/flaky tests - Mock asyncio.open_connection in test_async_integration.py error tests - Remove section comment headers (TEST-XXX) from test_client_methods.py - Add comment explaining setuptools dependency for PyCharm * test: add integration test coverage and docker marker - Add 'docker' marker to pyproject.toml for Docker-dependent tests - Add reqmod() integration tests for sync and async clients - Add async scan_stream() integration tests - Add async respmod() with preview tests (small/large/eicar) - Add connection reuse tests for sync and async clients - Add SSL tests for scan_file, scan_stream, and async respmod - Mark all Docker-dependent tests with @pytest.mark.docker - Remove redundant comments from pycap/ library code --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Jason Ahlstrand <[email protected]>
1 parent 08fcc1b commit 501b13e

44 files changed

Lines changed: 13884 additions & 571 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/codeql.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: CodeQL
2+
3+
on:
4+
push:
5+
branches: [ main, master, develop ]
6+
pull_request:
7+
branches: [ main, master, develop ]
8+
schedule:
9+
- cron: '0 6 * * 1' # Weekly on Mondays at 6am UTC
10+
11+
permissions:
12+
security-events: write
13+
contents: read
14+
actions: read
15+
16+
jobs:
17+
analyze:
18+
runs-on: ubuntu-latest
19+
20+
steps:
21+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
22+
23+
- name: Initialize CodeQL
24+
uses: github/codeql-action/init@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4
25+
with:
26+
languages: python
27+
28+
- name: Perform CodeQL Analysis
29+
uses: github/codeql-action/analyze@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4

‎.github/workflows/lint.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: Lint and Format
2+
3+
on:
4+
push:
5+
branches: [ main, master, develop ]
6+
pull_request:
7+
branches: [ main, master, develop ]
8+
9+
permissions:
10+
contents: read
11+
12+
jobs:
13+
ruff:
14+
runs-on: ubuntu-latest
15+
16+
steps:
17+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
18+
19+
- name: Install uv
20+
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
21+
22+
- name: Set up Python
23+
run: uv python install 3.12
24+
25+
- name: Run ruff linter
26+
run: uv run ruff check .
27+
28+
- name: Run ruff formatter check
29+
run: uv run ruff format --check .

‎.github/workflows/test.yml‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [ main, master, develop ]
6+
pull_request:
7+
branches: [ main, master, develop ]
8+
workflow_dispatch:
9+
inputs:
10+
run_integration:
11+
description: 'Run integration tests'
12+
required: false
13+
default: 'false'
14+
type: boolean
15+
16+
permissions:
17+
contents: read
18+
19+
jobs:
20+
unit-tests:
21+
runs-on: ubuntu-latest
22+
strategy:
23+
matrix:
24+
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
25+
26+
steps:
27+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
28+
29+
- name: Install uv
30+
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
31+
32+
- name: Set up Python ${{ matrix.python-version }}
33+
run: uv python install ${{ matrix.python-version }}
34+
35+
- name: Install dependencies
36+
run: uv sync --all-extras
37+
38+
- name: Run unit tests
39+
run: uv run pytest -v -m "not integration"
40+
41+
integration-tests:
42+
runs-on: ubuntu-latest
43+
if: ${{ github.event_name == 'workflow_dispatch' && inputs.run_integration == 'true' }}
44+
45+
steps:
46+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
47+
48+
- name: Install uv
49+
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
50+
51+
- name: Set up Python
52+
run: uv python install 3.12
53+
54+
- name: Install dependencies
55+
run: uv sync --all-extras
56+
57+
- name: Build and start ICAP server
58+
run: |
59+
docker compose -f docker/docker-compose.yml up -d
60+
sleep 30 # Wait for services to initialize
61+
62+
- name: Run integration tests
63+
run: uv run pytest -v -m integration
64+
65+
- name: Stop ICAP server
66+
if: always()
67+
run: docker compose -f docker/docker-compose.yml down

‎.github/workflows/typecheck.yml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: Type Check
2+
3+
on:
4+
push:
5+
branches: [ main, master, develop ]
6+
pull_request:
7+
branches: [ main, master, develop ]
8+
9+
permissions:
10+
contents: read
11+
12+
jobs:
13+
typecheck:
14+
runs-on: ubuntu-latest
15+
16+
steps:
17+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
18+
19+
- name: Install uv
20+
uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5
21+
22+
- name: Set up Python and install dependencies
23+
run: |
24+
uv python install 3.12
25+
uv sync --group dev
26+
27+
- name: Run type checker
28+
run: uv run ty check pycap

0 commit comments

Comments
 (0)