Skip to content

fix(litellm): close shared cloud client - #2903

Merged
chopratejas merged 1 commit into
headroomlabs-ai:mainfrom
abhinavkr26104:fix/2894-litellm-client-close
Aug 11, 2026
Merged

chopratejas merged 1 commit into
headroomlabs-ai:mainfrom
abhinavkr26104:fix/2894-litellm-client-close

Conversation

@abhinavkr26104

@abhinavkr26104 abhinavkr26104 commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds an explicit, idempotent async cleanup lifecycle for the LiteLLM callback's shared cloud HTTP client.

Fixes #2894

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • Added HeadroomCallback.aclose() to close the lazily-created httpx.AsyncClient and clear its reference.
  • Made cleanup safe when cloud mode was never used and when shutdown cleanup is invoked more than once.
  • Added regression coverage for initialized-client cleanup, reference clearing, and repeated/no-op cleanup.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

python -m pytest -q tests/test_integrations/test_litellm_callback.py
5 passed

ruff check .
All checks passed!

ruff format --check .
1382 files already formatted

python -m mypy headroom
Success: no issues found in 515 source files

python -m pytest -q
Collection blocked in this Windows environment by 174 errors, primarily missing compiled headroom._core; one unrelated test also lacks respx.

Real Behavior Proof

  • Environment: Windows, Python 3.12, loopback HTTP server, real httpx.AsyncClient.
  • Exact command / steps: Started a local HTTP server, configured HeadroomCallback(api_key="hdr_test", api_url="http://127.0.0.1:<port>"), ran _cloud_compress() against it, saved the created client, awaited callback.aclose(), then awaited callback.aclose() again.
  • Observed result: The real cloud request succeeded; the client was open during the request, reported closed after aclose(), the callback reference became None, and repeated cleanup was harmless.
  • Who maintains it: Headroom Labs maintains this active upstream repository and its LiteLLM integration.
  • Install surface: No dependencies or install behavior changed. Cloud mode continues to use the existing optional httpx dependency; no native code or runtime network access is introduced by this fix.
  • Not tested: The complete test suite could not run past collection because the local Windows environment lacks the compiled headroom._core extension.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • Documentation changes are not required; aclose() is documented in its public docstring and the host owns shutdown sequencing
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; targeted tests pass)
  • I did not edit CHANGELOG.md - it is generated by release-please from my Conventional Commit PR title.

Screenshots (if applicable)

Not applicable.

Additional Notes

The callback exposes aclose() for the host application's async shutdown lifecycle, matching the existing ASGI integration pattern.

@github-actions

github-actions Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

PR governance

This PR does not yet satisfy the required template fields:

  • Missing required section Description.

Please update the PR body, or move the PR back to draft while it is still in progress.

@github-actions github-actions Bot added the status: needs author action Pull request body or readiness checklist still needs author updates label Aug 10, 2026

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aclose() atomically detaches the shared client before awaiting closure, making repeated and concurrent cleanup calls idempotent and allowing the underlying async transport to drain correctly. No client is created on an unused/no-cloud path. The host-owned shutdown requirement is explicit in the API contract and tests cover initialized and no-op cases. The PR description checklist was corrected to reflect that no separate documentation file changed; broader CI has not run.

@chopratejas chopratejas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review complete: aclose atomically detaches the shared client before awaiting closure, making no-op, repeated, and concurrent cleanup safe under the application-owned lifecycle contract. Merge after full CI. Public lifecycle documentation and a real-client regression remain useful nonblocking follow-ups.

@chopratejas
chopratejas merged commit 07d89a7 into headroomlabs-ai:main Aug 11, 2026
71 of 73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs author action Pull request body or readiness checklist still needs author updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] LiteLLM cloud callback never closes its shared AsyncClient

3 participants