You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit acc8121
Browse filesBrowse the repository at this point in the historyBrowse files
In December 2023 we enabled - experimentally - the requirement to resolve all the open conversations in a
107
107
PR in order to make it merge-able. You will see in the status of the PR that it needs to have all the
@@ -121,14 +121,14 @@ already solved (assuming that maintainers will start treating the conversations
121
121
.. _coding_style:
122
122
123
123
Coding style and best practices
124
-
===============================
124
+
-------------------------------
125
125
126
126
Most of our coding style rules are enforced programmatically by ruff and mypy, which are run automatically
127
127
with static checks and on every Pull Request (PR), but there are some rules that are not yet automated and
128
128
are more Airflow specific or semantic than style.
129
129
130
130
Don't Use Asserts Outside Tests
131
-
-------------------------------
131
+
...............................
132
132
133
133
Our community agreed that to various reasons we do not use ``assert`` in production code of Apache Airflow.
134
134
For details check the relevant `mailing list thread <https://lists.apache.org/thread.html/bcf2d23fcd79e21b3aac9f32914e1bf656e05ffbcb8aa282af497a2d%40%3Cdev.airflow.apache.org%3E>`_.
@@ -155,7 +155,7 @@ The one exception to this is if you need to make an assert for type checking (wh
155
155
156
156
157
157
Database Session Handling
158
-
-------------------------
158
+
.........................
159
159
160
160
**Explicit is better than implicit.** If a function accepts a ``session`` parameter it should not commit the
161
161
transaction itself. Session management is up to the caller.
@@ -202,7 +202,7 @@ compatibility considerations. In most cases, ``session`` argument should be last
202
202
203
203
204
204
Don't use time() for duration calculations
205
-
-----------------------------------------
205
+
..........................................
206
206
207
207
If you wish to compute the time difference between two events with in the same process, use
208
208
``time.monotonic()``, not ``time.time()`` nor ``timezone.utcnow()``.
@@ -242,6 +242,42 @@ If the start_date of a duration calculation needs to be stored in a database, th
242
242
datetime objects. In all other cases, using datetime for duration calculation MUST be avoided as creating and
243
243
diffing datetime operations are (comparatively) slow.
244
244
245
+
Templated fields in Operator's __init__ method
246
+
..............................................
247
+
248
+
Airflow Operators might have some fields added to the list of ``template_fields``. Such fields should be
249
+
set in the constructor (``__init__`` method) of the operator and usually their values should
250
+
come from the ``__init__`` method arguments. The reason for that is that the templated fields
251
+
are evaluated at the time of the operator execution and when you pass arguments to the operator
252
+
in the DAG, the fields that are set on the class just before the ``execute`` method is called
253
+
are processed through templating engine and the fields values are set to the result of applying the
254
+
templating engine to the fields (in case the field is a structure such as dict or list, the templating
255
+
engine is applied to all the values of the structure).
256
+
257
+
That's why we expect two things in case of ``template fields``:
258
+
259
+
* with a few exceptions, only self.field = field should be happening in the operator's constructor
260
+
* validation of the fields should be done in the ``execute`` method, not in the constructor because in
261
+
the constructor, the field value might be a templated value, not the final value.
262
+
263
+
The exceptions are cases where we want to assign empty default value to a mutable field (list or dict)
264
+
or when we have a more complex structure which we want to convert into a different format (say dict or list)
265
+
but where we want to keep the original strings in the converted structure.
266
+
267
+
In such cases we can usually do something like this
0 commit comments