You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9e841f2
Browse filesBrowse the repository at this point in the historyBrowse files
feat: bind number and true/false loop variables in generated Python operators
A number or true/false property of an operator whose Scala descriptor
generates Python can now refer to a loop variable inside a control
block. The code is generated before the loop runs, so
PythonOperatorDescriptor.loopVariableBinding generates it from copies of
the descriptor with probe values in the reference's place: 3 and 5, then
70 and 90 for an integer; 0.25 and 0.75, then 2.5 and 7.5 for a number;
True and False. Where the two codes differ by exactly the two values,
and nowhere else, the code calls self.loop_variable_value(name, kind)
there instead: 'integer' for an int property, 'long' for one that takes
a whole number past 32 bits. The value must also land as it is at 0, 1,
-1, 1000000 and the other probes. When no probe pair generates the
code, the properties' defaults and the values next to them are tried,
so a descriptor's own check never reports a probe value. A value the
descriptor checks or adjusts, one that lands in a text, a comment or a
name, one the code computes with, never holds or reads when the
operator starts (in open or __init__, before any state message), and
one that decides the output schema leave the reference a compile error
that names the property and why.
WorkflowCompiler probes the code once per compile: the physical
operator runs that binding's code, and unbindableStateReferences
decides every reason from the same binding, the code's before the
schema's, and the schema one only once every input has a schema, as
the schema propagation does.
Operator.loop_variable_value reads the variable's text as
loop_variable_text spells it, from the values bound at the operator's
first tuple, and converts it as a JVM operator's property binds it
(StateReferenceBinding.coerce): trimmed as String.trim, an integer as
new BigDecimal(text).longValueExact() reads it, and within 32 bits for
an int property, as Jackson writes it there; a number as
Double.parseDouble does (2.5d and 0x1p3, but not inf, nan, 1_000 or
fullwidth digits), and a boolean as true or false in any case. A whole
double past 10^16 reads as the whole number its fewest digits spell, as
JDK 19 and later bind it; JDK 17 spells it with other digits
(JDK-4511638). An integer past 64 bits is refused, as
loop_variable_text refuses it.
The executor manager also hands each Python operator the values its
code reads through loop_variable_value, so binding fails, naming each,
for a required value no state message carried or one its property
cannot take, as the JVM's binding does, whether or not the code reads
the value then.
0 commit comments