Skip to content

Add support for arrays of different data types in the Trigger Form UI#32734

Merged
eladkal merged 7 commits into
apache:mainfrom
MatthieuBlais:feature/complex_arrays_for_trigger_forms
Aug 3, 2023
Merged

Add support for arrays of different data types in the Trigger Form UI#32734
eladkal merged 7 commits into
apache:mainfrom
MatthieuBlais:feature/complex_arrays_for_trigger_forms

Conversation

@MatthieuBlais

Copy link
Copy Markdown
Contributor

Description

The new Trigger Form UI released in 2.6 is handy but doesn't support array params of data types other than string. For more flexibility, this PR proposes to generate a JSON field similar to the object param if the users specify the attribute items in schema.

How to reproduce the issue

Use an array param and require the items to be integers.

with DAG(
    dag_id=Path(__file__).stem,
    description=__doc__[0 : __doc__.find(".")],
    doc_md=__doc__,
    schedule=None,
    start_date=datetime.datetime(2022, 3, 4),
    catchup=False,
    tags=["example_ui"],
    params={
        "array_of_numbers": Param(
            [1, 2, 3],
            "Only integers are accepted in this array",
            type="array",
            title="Array of numbers",
            items={"type": "number"},
        ),
    },
) as dag:

    @task(task_id="show_params")
    def show_params(**kwargs) -> None:
        ti: TaskInstance = kwargs["ti"]
        dag_run: DagRun = ti.dag_run
        if not dag_run.conf:
            print("Uups, no parameters supplied as DagRun.conf, was the trigger w/o form?")
            raise AirflowSkipException("No DagRun.conf parameters supplied.")
        print(f"This DAG was triggered with the following parameters:\n{json.dumps(dag_run.conf, indent=4)}")

    show_params()

The Generated Conf JSON shows an array of strings

image

And we cannot trigger the DAG:

image

Workaround

We can manually update the Generated Configuration JSON box, but it is not user friendly (especially if we expect an array of objects!), and will become counter intuitive as users get used to the benefits of the new trigger form.

What do you think @jens-scheffler-bosch ?

@boring-cyborg boring-cyborg Bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues kind:documentation labels Jul 21, 2023
@jscheffl jscheffl self-assigned this Jul 21, 2023
@jscheffl jscheffl added type:improvement Changelog: Improvements and removed kind:documentation labels Jul 21, 2023

@jscheffl jscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some small beautification proposals in documentation, otherwise - I like it! Thanks for the contribution!
(Take my approval emotional, I have no committer status)

Comment thread docs/apache-airflow/core-concepts/params.rst Outdated
Comment thread airflow/example_dags/example_params_ui_tutorial.py Outdated
@jscheffl jscheffl added this to the Airflow 2.7.0 milestone Jul 22, 2023
@MatthieuBlais
MatthieuBlais force-pushed the feature/complex_arrays_for_trigger_forms branch from 9ed9cf4 to f029ad1 Compare July 23, 2023 15:00
@MatthieuBlais

Copy link
Copy Markdown
Contributor Author

Great, thanks for the feedback! I've updated the PR with your commits

@jscheffl

Copy link
Copy Markdown
Contributor

Great, thanks for the feedback! I've updated the PR with your commits

Uuups, sorry, my comments were not syntax checked :-( The URL is not correctly formed to be accepted by Sphinx.

@MatthieuBlais
MatthieuBlais force-pushed the feature/complex_arrays_for_trigger_forms branch from 496b0a1 to 808d2bb Compare July 24, 2023 15:00
@eladkal
eladkal merged commit da69315 into apache:main Aug 3, 2023
@ephraimbuddy ephraimbuddy removed this from the Airflow 2.7.1 milestone Aug 3, 2023
@ephraimbuddy ephraimbuddy added this to the Airflow 2.7.0 milestone Aug 3, 2023
ephraimbuddy pushed a commit that referenced this pull request Aug 3, 2023
…#32734)

* Add JSON field for complex arrays with types other than string

---------

Co-authored-by: Matthieu Blais <[email protected]>
Co-authored-by: Jens Scheffler <[email protected]>
Co-authored-by: eladkal <[email protected]>
(cherry picked from commit da69315)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues type:improvement Changelog: Improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants