Skip to content

DX-2386: Revert httpbin test-infra migration to pre-migration baseline (5.14.0)#8450

Merged
buger merged 6 commits into
release-5.14.0from
revert-httpbin-migration-release-5.14.0
Jul 6, 2026
Merged

DX-2386: Revert httpbin test-infra migration to pre-migration baseline (5.14.0)#8450
buger merged 6 commits into
release-5.14.0from
revert-httpbin-migration-release-5.14.0

Conversation

@probelabs

@probelabs probelabs Bot commented Jul 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@probelabs
probelabs Bot requested review from a team as code owners July 6, 2026 09:11
@probelabs

probelabs Bot commented Jul 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

This pull request reverts the test infrastructure's httpbin service from a self-hosted mccutchen/go-httpbin image back to the original kennethreitz/httpbin image. This action is intended to resolve issues introduced during a recent migration and restore stability to the CI pipeline by reverting the changes from PRs #8418, #8428, and #8440.

Files Changed Analysis

The changes are consistently applied across 26 files, primarily within the CI/CD and testing infrastructure:

  • GitHub Workflows (.github/workflows/release.yml): The httpbin.org service image is changed back to kennethreitz/httpbin.
  • Docker Compose Files (ci/**/docker-compose.yml): Multiple Docker Compose files used for various test suites (API functionality, metrics, tracing, plugins) are updated to use the kennethreitz/httpbin image. The service name is also updated from httpbin to httpbin.org.
  • Test Configurations (.json, .yml): Numerous test API definitions and tracing scenarios have their target_url and other endpoint references updated from the internal service name (http://httpbin/...) to the new service name (http://httpbin.org/...).

Architecture & Impact Assessment

  • What this PR accomplishes: It rolls back a recent change to a core test dependency, prioritizing CI pipeline stability over the use of a self-hosted httpbin service.
  • Key technical changes introduced: The primary change is the replacement of the mccutchen/go-httpbin Docker image (which was pinned to a specific digest) with the unpinned kennethreitz/httpbin image. Correspondingly, the service name and all URLs in test configurations have been updated from httpbin to httpbin.org.
  • Affected system components: The impact is strictly limited to the CI/CD testing environment. No production code is affected. The change affects test suites for API functionality, metrics, tracing, and various plugins.
graph TD
    subgraph "Previous State"
        A[CI Test Jobs] --> B["Service: httpbin<br>Image: mccutchen/go-httpbin@sha256..."]
    end

    subgraph "After this PR"
        C[CI Test Jobs] --> D[Service: httpbin.org<br>Image: kennethreitz/httpbin]
    end
Loading

Scope Discovery & Context Expansion

The scope of this PR is narrowly focused on CI/CD configuration. By reverting three previous pull requests, it undoes a recent migration of the httpbin test dependency. The motivation is to resolve functional instabilities discovered in the go-httpbin implementation that were affecting automated tests. While this revert solves an immediate problem, it re-introduces a dependency on an external public service and removes version pinning, which can be a source of test flakiness due to network latency, rate limiting, or upstream changes. The impact is contained within the development lifecycle, aiming to restore reliability to test runs.

Metadata
  • Review Effort: 1 / 5
  • Primary Label: chore

Powered by Visor from Probelabs

Last updated: 2026-07-06T11:27:53.680Z | Triggered by: pr_updated | Commit: 27fe975

💡 TIP: You can chat with Visor using /visor ask <your question>

@probelabs

probelabs Bot commented Jul 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Security Issues (8)

Severity Location Issue
🟡 Warning .github/workflows/release.yml:1250
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning .github/workflows/release.yml:1302
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/smoke-tests/plugin-aliasing/docker-compose.yml:23
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/api-functionality/docker-compose.yml:37
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/metrics/docker-compose.yml:44
The Docker image `kennethreitz/httpbin` is pinned to the mutable `latest` tag instead of a specific digest. Using a mutable tag for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/plugin-compiler/docker-compose.yml:38
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/python-plugins/docker-compose.yml:47
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/tracing/docker-compose.yml:124
The Docker image `kennethreitz/httpbin` is pinned to the mutable `latest` tag instead of a specific digest. Using a mutable tag for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.

Architecture Issues (1)

Severity Location Issue
🟡 Warning .github/workflows/release.yml:1249
This PR reverts to using the public `kennethreitz/httpbin` Docker image, but it uses a floating tag (`:latest`, either explicitly or implicitly). This re-introduces a risk of CI pipeline instability and non-reproducible builds if the upstream image changes unexpectedly. The previous approach of pinning the dependency to a specific SHA256 digest is an architectural best practice for ensuring build stability and should be restored.
💡 SuggestionTo mitigate this risk, find a stable version of `kennethreitz/httpbin` and pin it to its SHA256 digest. This should be applied consistently across all configuration files (e.g., `docker-compose.yml` files in `ci/` and this workflow file). If a specific digest is problematic, pinning to a specific version tag is the next best alternative.

Security Issues (8)

Severity Location Issue
🟡 Warning .github/workflows/release.yml:1250
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning .github/workflows/release.yml:1302
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/smoke-tests/plugin-aliasing/docker-compose.yml:23
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/api-functionality/docker-compose.yml:37
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/metrics/docker-compose.yml:44
The Docker image `kennethreitz/httpbin` is pinned to the mutable `latest` tag instead of a specific digest. Using a mutable tag for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/plugin-compiler/docker-compose.yml:38
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/python-plugins/docker-compose.yml:47
The Docker image `kennethreitz/httpbin` is not pinned to a specific digest. Using a mutable tag (like `latest`, which is the default) for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
🟡 Warning ci/tests/tracing/docker-compose.yml:124
The Docker image `kennethreitz/httpbin` is pinned to the mutable `latest` tag instead of a specific digest. Using a mutable tag for a dependency in a CI/CD pipeline introduces a supply-chain risk. A malicious or broken image could be pushed to this tag, compromising the build environment. The previous configuration correctly used a pinned SHA256 digest, which is a security best practice.
💡 SuggestionPin the Docker image to a specific, known-good SHA256 digest to ensure build reproducibility and mitigate supply-chain attacks. For example: `image: kennethreitz/httpbin@sha256:f234b527c2565c5890115aeda53b1b5a26a3504f795350c62001c649c9514e38`.
\n\n ### Architecture Issues (1)
Severity Location Issue
🟡 Warning .github/workflows/release.yml:1249
This PR reverts to using the public `kennethreitz/httpbin` Docker image, but it uses a floating tag (`:latest`, either explicitly or implicitly). This re-introduces a risk of CI pipeline instability and non-reproducible builds if the upstream image changes unexpectedly. The previous approach of pinning the dependency to a specific SHA256 digest is an architectural best practice for ensuring build stability and should be restored.
💡 SuggestionTo mitigate this risk, find a stable version of `kennethreitz/httpbin` and pin it to its SHA256 digest. This should be applied consistently across all configuration files (e.g., `docker-compose.yml` files in `ci/` and this workflow file). If a specific digest is problematic, pinning to a specific version tag is the next best alternative.
\n\n ### Performance Issues (1)
Severity Location Issue
🟡 Warning .github/workflows/release.yml:1250
The `httpbin` Docker image is referenced using an implicit or explicit `:latest` tag across multiple CI configuration files (e.g., `.github/workflows/release.yml`, `ci/**/docker-compose.yml`). This can lead to unpredictable CI behavior when the image is updated and can degrade performance by causing CI runners to miss caches and re-download the image. The reverted code used a pinned digest, which is a best practice for ensuring reproducible and performant builds.
💡 SuggestionPin the `kennethreitz/httpbin` image to a specific digest in all CI configuration files where it is used. This will ensure build stability and improve performance by leveraging Docker's layer caching. For example: `image: kennethreitz/httpbin@sha256:22591433917951333505358253cb452144094379a4785420cec40268854a3793`.

Quality Issues (7)

Severity Location Issue
🟡 Warning .github/workflows/release.yml:1249
The Docker image `kennethreitz/httpbin` is referenced using a floating tag (`latest` is implied). This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).
🟡 Warning .github/workflows/release.yml:1308
The Docker image `kennethreitz/httpbin` is referenced using a floating tag (`latest` is implied). This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).
🟡 Warning ci/tests/api-functionality/docker-compose.yml:37
The Docker image `kennethreitz/httpbin` is referenced using a floating tag (`latest` is implied). This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).
🟡 Warning ci/tests/metrics/docker-compose.yml:45
The Docker image `kennethreitz/httpbin:latest` is referenced using a floating tag. This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).
🟡 Warning ci/tests/plugin-compiler/docker-compose.yml:38
The Docker image `kennethreitz/httpbin` is referenced using a floating tag (`latest` is implied). This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).
🟡 Warning ci/tests/python-plugins/docker-compose.yml:47
The Docker image `kennethreitz/httpbin` is referenced using a floating tag (`latest` is implied). This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).
🟡 Warning ci/tests/tracing/docker-compose.yml:125
The Docker image `kennethreitz/httpbin:latest` is referenced using a floating tag. This can lead to non-reproducible builds and unexpected failures in the CI pipeline if the image is updated with breaking changes.
💡 SuggestionTo ensure build stability and reproducibility, pin the image to a specific digest (SHA256 hash). Find a stable version of `kennethreitz/httpbin` that works for the tests, and pin it using its digest. For example: `image: kennethreitz/httpbin@sha256:f88953f72c23185590901b2036048054359392cddb43d31cef5f041188353d9e` (this is an example digest, please use the correct one for a known-good version).

Powered by Visor from Probelabs

Last updated: 2026-07-06T11:27:27.453Z | Triggered by: pr_updated | Commit: 27fe975

💡 TIP: You can chat with Visor using /visor ask <your question>

@sentinelone-cnapp-eu1

sentinelone-cnapp-eu1 Bot commented Jul 6, 2026

Copy link
Copy Markdown

SentinelOne CNS Hardcoded Secret Detector
✅ Congratulations, your code is safe

SentinelOne CNS is a cloud-agnostic, agentless CSPM & CWPP solution that continuously detects and prevents vulnerabilities that have the highest probability of being exploited in Azure, AWS, Google Cloud, and Kubernetes.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🚨 Jira Linter Failed

Commit: 27fe975
Failed at: 2026-07-06 11:26:06 UTC

The Jira linter failed to validate your PR. Please check the error details below:

🔍 Click to view error details 
failed to validate branch and PR title rules: branch name 'revert-httpbin-migration-release-5.14.0' must contain a valid Jira ticket ID (e.g., ABC-123)

Next Steps

  • Ensure your branch name contains a valid Jira ticket ID (e.g., ABC-123)
  • Verify your PR title matches the branch's Jira ticket ID
  • Check that the Jira ticket exists and is accessible

This comment will be automatically deleted once the linter passes.

@buger
buger enabled auto-merge (squash) July 6, 2026 11:54
@sonarqubecloud

sonarqubecloud Bot commented Jul 6, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
1 Security Hotspot

See analysis details on SonarQube Cloud

@buger
buger merged commit ade1179 into release-5.14.0 Jul 6, 2026
49 of 51 checks passed
@buger
buger deleted the revert-httpbin-migration-release-5.14.0 branch July 6, 2026 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@buger