Skip to content

fix: handle transient network errors in PyPI validation test#295

Merged
peterj merged 1 commit intoagentregistry-dev:mainfrom
optimus-fulcria:fix/flaky-pypi-test-194
Mar 11, 2026
Merged

fix: handle transient network errors in PyPI validation test#295
peterj merged 1 commit intoagentregistry-dev:mainfrom
optimus-fulcria:fix/flaky-pypi-test-194

Conversation

@optimus-fulcria
Copy link
Copy Markdown
Contributor

@optimus-fulcria optimus-fulcria commented Mar 6, 2026

Description

  • Motivation: The PyPI validation tests fail intermittently in CI when PyPI is unreachable (timeout, DNS failure). Fixes Flake: TestValidatePyPI_RealPackages/non-existent_package_should_fail #194.
  • What changed:
    • Added networkBound and allowNetError flags to test cases that depend on PyPI being reachable
    • Non-existent package test accepts network errors as valid failures (both prove the package cannot be validated)
    • Other network-bound tests skip gracefully on transient errors instead of failing
    • Added isNetworkError helper that checks for common transient error patterns (timeout, DNS, connection refused, TLS handshake)

Change Type

/kind flake

Changelog

Handle transient network errors in PyPI validation tests to reduce CI flakiness

Additional Notes

  • Only internal/registry/validators/registries/pypi_test.go is changed.

@peterj
Copy link
Copy Markdown
Contributor

peterj commented Mar 9, 2026

can you update the PR template

Copy link
Copy Markdown
Contributor

@peterj peterj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please update the pr template

Network-bound test cases now handle transient errors (timeout, DNS
failure, connection refused) gracefully instead of failing the test
suite. Non-existent package tests accept network errors as valid
failures since both outcomes prove the package cannot be validated.

Fixes agentregistry-dev#194

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@github-actions
Copy link
Copy Markdown

You already have 3 pull requests open. Please consider working on getting the existing ones merged before opening new ones. Thanks!

1 similar comment
@github-actions
Copy link
Copy Markdown

You already have 3 pull requests open. Please consider working on getting the existing ones merged before opening new ones. Thanks!

@optimus-fulcria
Copy link
Copy Markdown
Contributor Author

Updated the PR template and rebased on latest main. Only pypi_test.go is changed now.

@optimus-fulcria
Copy link
Copy Markdown
Contributor Author

PR template has been updated to follow the standard format (Description, Change Type, Changelog, Additional Notes). Ready for re-review.

@github-actions
Copy link
Copy Markdown

You already have 3 pull requests open. Please consider working on getting the existing ones merged before opening new ones. Thanks!

@github-actions
Copy link
Copy Markdown

You already have 3 pull requests open. Please consider working on getting the existing ones merged before opening new ones. Thanks!

@optimus-fulcria
Copy link
Copy Markdown
Contributor Author

Updated PR description to match the template. Ready for re-review.

@peterj peterj added this pull request to the merge queue Mar 11, 2026
Merged via the queue into agentregistry-dev:main with commit 613a378 Mar 11, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flake: TestValidatePyPI_RealPackages/non-existent_package_should_fail

2 participants