perf: use map instead of generator by henryiii · Pull Request #996 · pypa/packaging

henryiii · 2025-11-27T05:54:32Z

Another slow line in the profile. This one is the only case in the codebase that I see using a simple generator like this on .split("."), all the rest use map. Using map saves about 8% when creating Version's. Using tuple([...]) also saves around 8%, but I think the map is nicer than using a list comprehension inside a function call (and also consistent with the other cases, like in tags.py).

Edit: also found one more place, inside the __str__ method, which is used in SpecifierSet, for example.

henryiii · 2025-11-27T06:42:19Z

I also tried this variation:

diff --git a/src/packaging/version.py b/src/packaging/version.py
index ecaf6c6..db4e802 100644
--- a/src/packaging/version.py
+++ b/src/packaging/version.py
@@ -255,25 +255,11 @@ class Version(_BaseVersion):
         >>> str(Version("1.0a5"))
         '1.0a5'
         """
-        parts = [self.base_version]
-
-        # Pre-release
-        if self.pre is not None:
-            parts.append("".join(map(str, self.pre)))
-
-        # Post-release
-        if self.post is not None:
-            parts.append(f".post{self.post}")
-
-        # Development release
-        if self.dev is not None:
-            parts.append(f".dev{self.dev}")
-
-        # Local version segment
-        if self.local is not None:
-            parts.append(f"+{self.local}")
-
-        return "".join(parts)
+        pre = "" if self.pre is None else "".join(map(str, self.pre))
+        post = "" if self.post is None else f".post{self.post}"
+        dev = "" if self.dev is None else f".dev{self.dev}"
+        local = "" if self.local is None else f"+{self.local}"
+        return f"{self.base_version}{pre}{post}{dev}{local}"

     @property
     def epoch(self) -> int:

Which I think reads a little better, but it's not measurably faster (using str(Version(version))), in fact, it might be around 1% slower.

notatallshaw · 2025-11-27T07:07:17Z

I've also found places where using a map over a generator provides a small speed up, but I was hesitant to propose a PR because it wasn't clear to me if this would hold true in general or just happened to be true for the particular implementation of Python I was tested on.

Do you know, for example, if pypy is faster for maps or generators?

I'm not against this, especially when the map calls are so simple, I just want to make sure we're not chasing down implementation specific details that could change in the future.

henryiii · 2025-11-27T15:31:33Z

I've (mostly) been testing on the latest version of Python, which should have as many optimizations as possible for generators. List comprehensions are also fast, but map feels better than tuple([int(i) for i in match.group("release").split(".")]), which very much looks like an implementation optimization.

The map takes 8% less time on PyPy as well (PyPy is about 4x faster overall). Interestingly, older versions of Python have a smaller difference here; 3.8 using the map only saves 4%.

Signed-off-by: Henry Schreiner <[email protected]>

henryiii mentioned this pull request Nov 27, 2025

perf: better string building #997

Merged

henryiii force-pushed the henryiii/perf/map branch from a75335e to 041fd16 Compare November 27, 2025 15:19

notatallshaw approved these changes Nov 27, 2025

View reviewed changes

henryiii added 2 commits November 27, 2025 10:59

perf: use map instead of generator

ea9c6cf

Signed-off-by: Henry Schreiner <[email protected]>

perf: one more usage of map

50ee515

Signed-off-by: Henry Schreiner <[email protected]>

henryiii force-pushed the henryiii/perf/map branch from 041fd16 to 50ee515 Compare November 27, 2025 15:59

henryiii merged commit f3440e9 into pypa:main Nov 27, 2025
40 checks passed

henryiii deleted the henryiii/perf/map branch November 27, 2025 16:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: use map instead of generator#996

perf: use map instead of generator#996
henryiii merged 2 commits intopypa:mainfrom
henryiii:henryiii/perf/map

henryiii commented Nov 27, 2025 •

edited

Loading

Uh oh!

henryiii commented Nov 27, 2025

Uh oh!

notatallshaw commented Nov 27, 2025 •

edited

Loading

Uh oh!

henryiii commented Nov 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

henryiii commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

henryiii commented Nov 27, 2025

Uh oh!

notatallshaw commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

henryiii commented Nov 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

henryiii commented Nov 27, 2025 •

edited

Loading

notatallshaw commented Nov 27, 2025 •

edited

Loading