Skip to content

Edge: Return 400 instead of 500 on invalid worker version - #72925

Merged
jscheffl merged 6 commits into
apache:mainfrom
1fanwang:1fannnw/edge-worker-invalid-version-400
Nov 6, 2026
Merged

jscheffl merged 6 commits into
apache:mainfrom
1fanwang:1fannnw/edge-worker-invalid-version-400

Conversation

@1fanwang

@1fanwang 1fanwang commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

An Edge worker reports its Airflow and Edge provider versions when it registers. With a minimum acceptable worker version configured, the API server parses those versions, and a bad value from the worker returned HTTP 500 instead of a client error. Two inputs hit this: a string that is not a valid version, and a version sent as a number, which the request schema accepts.

The API now answers both with HTTP 400 and a message naming the bad value, as the route already documents. A bad minimum version in the server config still returns 500, because that is a server misconfiguration rather than bad worker input.

Testing Done

Registered workers through the real worker route with a minimum core version of 0.0.0, sending three bad core versions. Each run covers packaging 25.0 and the locked 26.3, because a numeric version fails differently on them: 26.2 and older raise TypeError, 26.3 raises InvalidVersion.

Before, with worker.py from apache/main:

git show apache/main:providers/edge3/src/airflow/providers/edge3/worker_api/routes/worker.py > providers/edge3/src/airflow/providers/edge3/worker_api/routes/worker.py
uv run --project providers/edge3 --with packaging==25.0 python repro_worker_version.py
uv run --project providers/edge3 python repro_worker_version.py
packaging 25.0
  'invalid-version': HTTP 500 Internal Server Error
  3: HTTP 500 Internal Server Error
  3.5: HTTP 500 Internal Server Error
packaging 26.3
  'invalid-version': HTTP 500 Internal Server Error
  3: HTTP 500 Internal Server Error
  3.5: HTTP 500 Internal Server Error

After, on this branch:

git checkout -- providers/edge3/src/airflow/providers/edge3/worker_api/routes/worker.py
uv run --project providers/edge3 --with packaging==25.0 python repro_worker_version.py
uv run --project providers/edge3 python repro_worker_version.py
packaging 25.0
  'invalid-version': HTTP 400 {"detail":"Invalid version string 'invalid-version': Invalid version: 'invalid-version'"}
  3: HTTP 400 {"detail":"Invalid version 3: expected a string, got int"}
  3.5: HTTP 400 {"detail":"Invalid version 3.5: expected a string, got float"}
packaging 26.3
  'invalid-version': HTTP 400 {"detail":"Invalid version string 'invalid-version': Invalid version: 'invalid-version'"}
  3: HTTP 400 {"detail":"Invalid version 3: expected a string, got int"}
  3.5: HTTP 400 {"detail":"Invalid version 3.5: expected a string, got float"}
Reproducer source: repro_worker_version.py
import os

os.environ["AIRFLOW__API_AUTH__JWT_SECRET"] = "repro-secret"
os.environ["AIRFLOW__EDGE__MINIMUM_ACCEPTABLE_CORE_VERSION_FOR_WORKERS"] = "0.0.0"

import packaging
from fastapi import FastAPI
from fastapi.testclient import TestClient

from airflow.api_fastapi.auth.tokens import JWTGenerator
from airflow.providers.edge3 import __version__ as edge_provider_version
from airflow.providers.edge3.worker_api.routes.worker import worker_router

app = FastAPI()
app.include_router(worker_router, prefix="/edge_worker/v1")
token = JWTGenerator(secret_key="repro-secret", valid_for=60, audience="api").generate(
    extras={"method": "worker/repro"}
)
print(f"packaging {packaging.__version__}")
with TestClient(app, raise_server_exceptions=False, headers={"Authorization": token}) as client:
    for worker_version in ("invalid-version", 3, 3.5):
        sysinfo = {"airflow_version": worker_version, "edge_provider_version": edge_provider_version}
        response = client.post("/edge_worker/v1/worker/repro", json={"state": "starting", "sysinfo": sysinfo})
        print(f"  {worker_version!r}: HTTP {response.status_code} {response.text}")

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: GitHub Copilot CLI (GPT-6 Astra) following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:providers provider:edge Edge Executor / Worker (AIP-69) / edge3 labels Sep 11, 2026
@1fanwang
1fanwang force-pushed the 1fannnw/edge-worker-invalid-version-400 branch from 877e978 to 86e4875 Compare September 11, 2026 17:47
@dheerajturaga

Copy link
Copy Markdown
Member

Thanks for the fix. Could you please address these items?

  • The provider mypy check fails at test_worker.py:114 because body["sysinfo"] is inferred as object. Please add explicit typing and rerun CI.
  • _version() catches InvalidVersion, but sysinfo also accepts integers, floats, and datetimes. Passing one of those to Version() raises TypeError, so malformed client input can still return HTTP 500. Please validate that version fields are strings or handle TypeError, and add a regression test.

Drafted-by: Codex (GPT-5); reviewed by @dheerajturaga before posting

This was referenced Sep 25, 2026
When minimum acceptable worker versions were configured, an edge worker
submitting an unparseable version string caused packaging.version.Version to
raise InvalidVersion. Because this exception was unhandled, it escaped as an
HTTP 500 Internal Server Error.

Catch InvalidVersion in _version() and return a clean 400 Bad Request error
matching the route error specification.

Signed-off-by: 1fanwang <[email protected]>
A malformed server minimum must not be reported as bad worker input. Cover both registration and heartbeat through authenticated HTTP requests.

Signed-off-by: 1fanwang <[email protected]>
The test body is a heterogeneous dict literal, so mypy infers its value type
as object and rejects the nested assignment into body["sysinfo"]. Building
sysinfo first keeps its dict[str, str | int | float | datetime] type while
the body still references the same object, so the mutation reaches the
request payload exactly as before.

Signed-off-by: 1fanwang <[email protected]>
The worker sysinfo payload accepts numbers as well as strings, so a worker
can report its version as 3 or 3.5. packaging releases before 26.3 raise
TypeError instead of InvalidVersion for such values, which still escaped as
HTTP 500 whenever a minimum acceptable worker version was configured.

Signed-off-by: 1fanwang <[email protected]>
@jscheffl
jscheffl merged commit 3e076e8 into apache:main Nov 6, 2026
72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:edge Edge Executor / Worker (AIP-69) / edge3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants