Issue32954
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-02-26 10:19 by arcivanov, last changed 2022-04-11 14:58 by admin.
| Messages (9) | |||
|---|---|---|---|
| msg312909 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:19 | |
I'd like to start a discussion on/gauge interest for introducing an enhancement to PEP-498 in a form of delayed/lazy/lambda f-string.
The proposed change is, conceptually, minor and would not differ from PEP-498 syntactically at all except for string prefix.
E.g.
extra = f'{extra},waiters:{len(self._waiters)}'
becomes
extra = fl'{extra},waiters:{len(self._waiters)}'
The proposal would produce a lambda-like object x('format'), where x.__str__() == f'format'.
This should come extremely useful in all cases where delayed or conditional string formatting and concatenation are desired, such as in cases of logging.
As an example, currently, the
logger.debug("Format %s string", value)
cannot be used with an f-string as follows
logger.debug(f"Format {value} string")
without an unconditional evaluation of all parameters due to current compilation prior to logging checking whether it's even enabled for debug:
>>> b = 1
>>> def a(x):
... return f"Foo {x} bar {b}"
...
>>> dis.dis(a)
2 0 LOAD_CONST 1 ('Foo ')
2 LOAD_FAST 0 (x)
4 FORMAT_VALUE 0
6 LOAD_CONST 2 (' bar ')
8 LOAD_GLOBAL 0 (b)
10 FORMAT_VALUE 0
12 BUILD_STRING 4
14 RETURN_VALUE
Additional great optimizations may be rendered by introducing an a fl-string is a case where
foo = "foo"
s1 = fl"S1 value ${foo}"
s2 = fl"S2 value of ${s1}"
print(s2)
may produce only one BUILD_STRING instruction, potentially dramatically increasing performance in heavy string-concat based applications.
Even when a compiler will not be able to statically prove that a particular value is in fact an fl-string, an interpreter-level check or a JIT-based Python implementation may be able to optimize such concats ad-nauseam (with trap-based deopt), allowing to collapse an extremely long chains of formats into a single BUILD_STRING op without any intermediary string allocation/concatenation (very useful in cases such as web servers, templating engines etc).
I'll be happy to produce patches against 3.7/3.8 for this if a general concept is found useful/desirable by the community.
|
|||
| msg312910 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * ![]() |
Date: 2018-02-26 10:27 | |
class FL:
def __init__(self, func):
self.func = func
def __str__(self):
return self.func()
extra = FL(lambda: f'{extra},waiters:{len(self._waiters)}')
|
|||
| msg312911 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:42 | |
As an example this is the current state of affairs:
>>> def x():
... foo = "foo"
... s1 = f"S1 value {foo}"
... s2 = f"S2 value {s1}"
... print(s2)
...
>>> dis.dis(x)
2 0 LOAD_CONST 1 ('foo')
2 STORE_FAST 0 (foo)
3 4 LOAD_CONST 2 ('S1 value ')
6 LOAD_FAST 0 (foo)
8 FORMAT_VALUE 0
10 BUILD_STRING 2
12 STORE_FAST 1 (s1)
4 14 LOAD_CONST 3 ('S2 value ')
16 LOAD_FAST 1 (s1)
18 FORMAT_VALUE 0
20 BUILD_STRING 2
22 STORE_FAST 2 (s2)
5 24 LOAD_GLOBAL 0 (print)
26 LOAD_FAST 2 (s2)
28 CALL_FUNCTION 1
30 POP_TOP
32 LOAD_CONST 0 (None)
34 RETURN_VALUE
whereas an fl-string representation of:
>>> def x():
... foo = "foo"
... s1 = fl"S1 value {foo}"
... s2 = fl"S2 value {s1}"
... print(s2)
...
would behave close to:
>>> def x():
... foo = "foo"
... s1 = lambda: f"S1 value {foo}"
... s2 = lambda: f"S2 value {s1()}"
... print(s2())
...
>>> dis.dis(x)
2 0 LOAD_CONST 1 ('foo')
2 STORE_DEREF 0 (foo)
3 4 LOAD_CLOSURE 0 (foo)
6 BUILD_TUPLE 1
8 LOAD_CONST 2 (<code object <lambda> at 0x7ff2ef1abd20, file "<stdin>", line 3>)
10 LOAD_CONST 3 ('x.<locals>.<lambda>')
12 MAKE_FUNCTION 8
14 STORE_DEREF 1 (s1)
4 16 LOAD_CLOSURE 1 (s1)
18 BUILD_TUPLE 1
20 LOAD_CONST 4 (<code object <lambda> at 0x7ff2ef1abae0, file "<stdin>", line 4>)
22 LOAD_CONST 3 ('x.<locals>.<lambda>')
24 MAKE_FUNCTION 8
26 STORE_FAST 0 (s2)
5 28 LOAD_GLOBAL 0 (print)
30 LOAD_FAST 0 (s2)
32 CALL_FUNCTION 0
34 CALL_FUNCTION 1
36 POP_TOP
38 LOAD_CONST 0 (None)
40 RETURN_VALUE
Where LOAD_CONST, LOAD_CONST, MAKE_FUNCTION sequences are replaced with BUILD_LAZY_STRING (name's provisional) producing the same lambda-like formatter function that cooperates with FORMAT_VALUE and BUILD_STRING/BUILD_LAZY_STRING ops.
|
|||
| msg312912 - (view) | Author: Eric V. Smith (eric.smith) * ![]() |
Date: 2018-02-26 10:51 | |
See also PEP 501. |
|||
| msg312913 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:54 | |
@serhiy.storchaka Of course a similar pattern can be implemented via a class (or even without one as I've shown below). But you can clearly notice that in your example: 1) There are tons of boilerplate (as in mine with lambdas). 2) It's going to be slower than a core implementation with specialized objects. 3) It can't be made to cooperate with a FORMAT_VALUE/BUILD_STRING - intermediate strings will still be produced. |
|||
| msg312914 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 10:59 | |
@eric.smith Thanks! I was looking for such a general-purpose proposal but could not find it. Although general-purpose mechanism that would allow pluggable constructs like `sh`, `html`, `sql` and the like is awesome and very desirable (especially sh in preference of Popen madness), string formatting/concatenation is IMO a fundamental concept (hence PEP-498 is in core with specialized opcodes instead of being deferred to be a general purpose pluggable PEP-501 implementation). |
|||
| msg312918 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 11:24 | |
> Although general-purpose mechanism that would allow pluggable constructs like `sh`, `html`, `sql` Strike that one, I didn't read into PEP-0501 deep enough before replying. Yes, `i"format"` is what I'm talking about. |
|||
| msg312922 - (view) | Author: Eric V. Smith (eric.smith) * ![]() |
Date: 2018-02-26 12:20 | |
Arcadiy: Somehow you're dropping Serhiy and me from the nosy list. I've re-added us. |
|||
| msg312923 - (view) | Author: Arcadiy Ivanov (arcivanov) | Date: 2018-02-26 12:46 | |
Sorry about that! I'll be sure to refresh next time before posting a reply. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:58 | admin | set | github: 77135 |
| 2019-12-18 06:25:31 | mbdevpl | set | nosy:
+ mbdevpl |
| 2018-02-26 16:10:28 | barry | set | nosy:
+ barry |
| 2018-02-26 12:46:52 | arcivanov | set | messages: + msg312923 |
| 2018-02-26 12:20:52 | eric.smith | set | nosy:
+ eric.smith, serhiy.storchaka messages: + msg312922 |
| 2018-02-26 11:24:30 | arcivanov | set | messages: + msg312918 |
| 2018-02-26 10:59:46 | arcivanov | set | messages: + msg312914 |
| 2018-02-26 10:54:18 | arcivanov | set | nosy:
- eric.smith, serhiy.storchaka messages: + msg312913 |
| 2018-02-26 10:51:24 | eric.smith | set | nosy:
+ eric.smith, serhiy.storchaka messages: + msg312912 |
| 2018-02-26 10:42:52 | arcivanov | set | nosy:
- serhiy.storchaka messages: + msg312911 components: + Interpreter Core, - Library (Lib) |
| 2018-02-26 10:27:49 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg312910 |
| 2018-02-26 10:19:19 | arcivanov | create | |
