Skip to content

AIP-84 Get Configs / Get Config Value#43841

Merged
pierrejeambrun merged 10 commits intoapache:mainfrom
jason810496:feature/AIP-84/add-config
Nov 20, 2024
Merged

AIP-84 Get Configs / Get Config Value#43841
pierrejeambrun merged 10 commits intoapache:mainfrom
jason810496:feature/AIP-84/add-config

Conversation

@jason810496
Copy link
Member

closes: #42745
related: #42370

@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. labels Nov 9, 2024
@jscheffl jscheffl added the legacy api Whether legacy API changes should be allowed in PR label Nov 9, 2024
@jscheffl jscheffl force-pushed the feature/AIP-84/add-config branch from e32d6a4 to 8387632 Compare November 9, 2024 18:00
@jason810496 jason810496 force-pushed the feature/AIP-84/add-config branch from 8387632 to a97c505 Compare November 10, 2024 06:24
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

A few suggestions but looking good.

@jason810496 jason810496 force-pushed the feature/AIP-84/add-config branch 2 times, most recently from 7d15731 to 8a97fd4 Compare November 15, 2024 18:25
@jason810496
Copy link
Member Author

Hi @pierrejeambrun, here is the update:

Common Accept Header for JSON and Text

I have added HeaderAcceptJsonOrText in common/headers.py, annotated with Mimetype from common/types.py.

I tested it with application/json; charset=utf-8, and for headers including utf-8, we need to use startswith to match the header correctly. Additionally, I included an OpenAPI schema for the Accept header.

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks looking good, a few suggestions / question, then we can merge.

@jason810496 jason810496 force-pushed the feature/AIP-84/add-config branch from 8a97fd4 to f24d243 Compare November 19, 2024 04:03
Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to merge after rebase

@jason810496 jason810496 force-pushed the feature/AIP-84/add-config branch from f24d243 to 41281dc Compare November 20, 2024 11:52
@jason810496
Copy link
Member Author

Just resolve the conflict 🚀

@pierrejeambrun
Copy link
Member

Hello @jason810496 do you mind fixing the formatting errors, it would be great to merge this one :)

@pierrejeambrun pierrejeambrun merged commit 3b8f834 into apache:main Nov 20, 2024
@bbovenzi
Copy link
Contributor

Screenshot 2024-11-20 at 11 32 21 AM

I was just testing this and noticed that we're passing all the values as strings. It would be nice if I am passing 'application/json' that values are actually converted so 'True' becomes true, '5' becomes 5

@rawwar
Copy link
Contributor

rawwar commented Nov 20, 2024

I was just testing this and noticed that we're passing all the values as strings. It would be nice if I am passing 'application/json' that values are actually converted so 'True' becomes true, '5' becomes 5

I think, that would require us to maintain a Model with all possible configs along with their type. From what I remember, we typecast them after retrieving from the config. For example, retries are fetched as a string and then typecasted in methods. Wrong example.

But, AirflowConfigParser has methods defined to typecast and reuse. Like for log_fetch_timeout

https://github.com/rawwar/airflow/blob/9f0c4dfe224fbc417eb107f04c19d7108341f839/airflow/utils/log/file_task_handler.py#L92

@bbovenzi , Should I create a separate issue? I don't think its a trivial change

@pierrejeambrun
Copy link
Member

pierrejeambrun commented Nov 20, 2024

I would be for creating an issue, and tagging that as a feature request ?

Unless this was already implemented in the legacy endpoint but I don't believe so.

That would be a great addition. Full string values is quite limited and force work on the client side.

@bbovenzi
Copy link
Contributor

I think I'll make this a feature for the UI only config endpoint

got686-yandex pushed a commit to got686-yandex/airflow that referenced this pull request Jan 30, 2025
* AIP-84 Get Config

* Fix section param type and text format

* Add test for Get Config

* Consolidate redundant for check expose config

* Add `Accept` header depends for Json and Text

* Refactor config, dag_source with common header

* Refactor test_config with non-sensitive-only case

* Fix OpenAPI schema for Headers, Get Config

* Refactor test_config

- add `HEADERS_NONE` and `HEADERS_ANY` cases
- modularize common json response cases
- add `_validate_response` common method

* Fix style by ruff format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:UI Related to UI/UX. For Frontend Developers. legacy api Whether legacy API changes should be allowed in PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AIP-84 Migrate Config related public endpoint to FastAPI

5 participants