Repository navigation
Commit e8ae850
authored
Reject empty match_glob in GCSToS3Operator on old google provider (#71591)
match_glob is a template field, so its value is not available until
Jinja rendering happens on the worker, well after __init__ has run. The
guard that rejects match_glob on an unsupported google provider tested
it for truthiness, which the validate-operators-init prek hook flags.
The check only asks whether the argument was supplied; it never
inspects the value. Per the false-positive guidance on #70296 that
makes it a provision check, which is fixed in place rather than moved
to execute(). Moving it would break the check under
render_template_as_native_obj=True, where a supplied field can render
to None and so becomes indistinguishable from an omitted one, and would
defer a static authoring mistake from Dag parse time to every task
instance and retry.
Switching to an explicit is not None comparison changes one case:
match_glob="" is now rejected on a google provider older than 10.3.0,
where truthiness previously read the supplied empty string as absent.
An empty glob is not a valid pattern and was never honoured on those
versions.
Narrow the raised exception from AirflowException to ValueError, which
is what an invalid argument warrants, and drop the file from
generated/known_airflow_exceptions.txt accordingly. This follows the
merged #70359 precedent for S3DeleteObjectsOperator, which made the
same one-for-one change to the same four kinds of file.
Also remove the class from the prek exemption list, since the hook
fails on stale entries, and add tests covering supplied, empty-string
and omitted match_glob.
Related: #702961 parent df9a3e7 commit e8ae850
4 files changed
Lines changed: 36 additions & 7 deletions
File tree
- generated
- providers/amazon
- src/airflow/providers/amazon/aws/transfers
- tests/unit/amazon/aws/transfers
- scripts/ci/prek
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
112 | 112 | | |
113 | 113 | | |
114 | 114 | | |
115 | | - | |
116 | 115 | | |
117 | 116 | | |
118 | 117 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
146 | 146 | | |
147 | 147 | | |
148 | 148 | | |
149 | | - | |
150 | | - | |
151 | | - | |
152 | | - | |
| 149 | + | |
| 150 | + | |
153 | 151 | | |
154 | 152 | | |
155 | 153 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
78 | 111 | | |
79 | 112 | | |
80 | 113 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
11 | 10 | | |
12 | 11 | | |
13 | 12 | | |
| |||
0 commit comments