Skip to content

Hide team scoped per-key secrets backend kwargs in config output - #77011

Open
ityashag wants to merge 1 commit into
apache:mainfrom
ityashag:fix-team-scoped-per-key-backend-kwargs-sensitivity
Open

ityashag wants to merge 1 commit into
apache:mainfrom
ityashag:fix-team-scoped-per-key-backend-kwargs-sensitivity

Conversation

@ityashag

@ityashag ityashag commented Nov 13, 2026 •

Copy link
Copy Markdown

A secrets backend's kwargs can also be set one key at a time (AIRFLOW__SECRETS__BACKEND_KWARG__<KEY>, and the [workers] equivalent). Those values carry the same credentials as backend_kwargs itself, such as a Vault role_id or secret_id. The config API redacted them by matching the literal secrets and workers section names in the API layer, so a team scoped spelling was not recognised: the [<team>=secrets] section, or AIRFLOW__<TEAM>___SECRETS__BACKEND_KWARG__*, which is reported under a section named after the team. As the issue notes, nothing leaks today because the secrets backend is not team aware, but the check fails open the moment it becomes so.

This recognises the per-key options in AirflowConfigParser.is_sensitive_option, next to the team scoped resolution added in #70755, so as_dict and both config endpoints apply one rule to every spelling. The API-only prefix list and helpers are removed, and mask_secrets reads the same list of backend kwargs options.

The per-key API tests mocked conf.as_dict to return unredacted values, so they only exercised the removed helper. They now set real environment variables and check the redaction end to end, for the base spellings and the team scoped one.

closes: #71037

Checks run:

  • uv run --project shared/configuration pytest shared/configuration/tests/configuration/test_parser.py: 70 passed. The two new tests fail against the parser on main.
  • uv run --project airflow-core pytest airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_config.py: 43 passed. With the source files from main, the team scoped cases fail in both endpoints.
  • prek run on the changed files, including mypy for shared-configuration and airflow-core: passed. Hooks that need breeze were skipped (SKIP_BREEZE_PREK_HOOKS), and breeze verify was not run.
  • Timing, since this is secret masking: conf.as_dict(display_sensitive=False) over 420 options went from 0.75 ms to 0.81 ms, and is_sensitive_option adds about 0.2 µs per call.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5.5)

Generated-by: Claude Code (Opus 5.5) following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

A secrets backend's kwargs can also be set one key at a time, and those
values carry the same credentials as backend_kwargs itself. The config API
recognised them only under the literal [secrets] and [workers] section
names, so a team scoped spelling would be returned in full by GET /config
once secrets backends become team aware.
@boring-cyborg

boring-cyborg Bot commented Nov 13, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: [email protected]
    Slack: https://s.apache.org/airflow-slack

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API need workflow approval

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Per-key secrets-backend config options are not recognised as sensitive under a team-scoped spelling

1 participant