colocate __main__.py with the central directory record of the zipapp#2209
colocate __main__.py with the central directory record of the zipapp#2209cosmicexplorer wants to merge 5 commits intopex-tool:mainfrom
Conversation
cosmicexplorer
force-pushed
the
move-main-to-end-of-sources
branch
2 times, most recently
from
5273031 to
9f19ad7
Compare
cosmicexplorer
force-pushed
the
move-main-to-end-of-sources
branch
3 times, most recently
from
5c617ff to
a89cc18
Compare
cosmicexplorer
force-pushed
the
move-main-to-end-of-sources
branch
from
a89cc18 to
84d9da8
Compare
cosmicexplorer
force-pushed
the
move-main-to-end-of-sources
branch
from
339a179 to
ae52c6b
Compare
cosmicexplorer
force-pushed
the
move-main-to-end-of-sources
branch
from
ae52c6b to
d125cd5
Compare
| dest = os.path.join(dirname, f) | ||
| safe_mkdir(os.path.dirname(dest)) | ||
| safe_copy(os.path.realpath(os.path.join(self._chroot.chroot, f)), dest) | ||
| with TRACER.timed("copying over uncached sources", V=9): |
There was a problem hiding this comment.
Ok, this seems fine - but boy oh boy does it seem incredibly complicated vs just ordering the labels correctly in the existing tuple on the LHS and writing a good block comment just above.
There was a problem hiding this comment.
You're right, and this change could be reduced quite a bit by providing the labels literally (for example, we could avoid changing this method at all). I thought it was useful to document the meanings of each label, since they're not really documented elsewhere unless you grep for the literal strings "executable", "importhook", etc. I was also thinking that naming each section with a classmethod would make it easier for someone to subclass PEXBuilder and insert their own files into the appropriate section while retaining the performance benefit for zipapps. If that's not a supported use case, we could very well reduce the size of this change by a lot.
|
@cosmicexplorer I'm a huge non-fan of Problem / Solution formulas for writing commit messages. We're adults and I think you can just write a good message. Quoting people - not helpful. Just explaining the change and its reason should stand on its own. |
Ok, I will keep that in mind for the future.
In this case it's not actually documented anywhere else that the python interpreter accesses |
|
Thanks @cosmicexplorer, yes. The facts remain the same regardless of who pointed them out. Much more sane to read in the git log. FWIW:
Is not true. When the Python interpreter executes a zipapp (or a directory) it only looks for |
|
You're right; I have fixed the wording to clarify that |
|
@cosmicexplorer since this is just over 1.5 years old, I'm going to close. You've lost interest and the speed gains are likely in the noise / I've lost interest in pursuing this nugget. |
2 participants
When the Python interpreter executes a zipapp, it will import
__main__.py, while__pex__/__init__.pywill be loaded if the pex file is on thePYTHONPATH. This file is loaded from the zip file handle withzipimport, and the PEX bootstrap script at that location will then import some of the code in.bootstrap/before the zipapp is unpacked and compiled. Because a zip file's central directory records are located at the end of the file, placing this code at the end of the zip file list will reduce the amount of seeking the Python interpreter needs to perform in order to execute the bootstrap script. This was previously discussed in #2175 (comment) and #2175 (comment).Changes involved:
Chroot#zip()respect the relative ordering of labels when collecting fromself.filesets."main"and"importhook"labels at the end of the zipapp.