Skip to content

Commit 2b0aa85

Browse files
committed
Import a lazy submodule that a package attribute shadows
_Py_module_getattro_impl consults lazy_pending_submodules only once ordinary lookup has missed, so a `sub` bound by pkg/__init__.py answers the read that was meant to trigger a pending `import pkg.sub`, and the submodule is never imported. An eager `import pkg.sub` replaces that attribute with the submodule, so the two modes disagree. Bind the submodule on the package instead, at the two points where the eager import would have: when the package finishes loading, and at registration when it is already loaded. _imp._set_lazy_attributes() now does what its name, and the comment on lazy_pending_submodules, already describe. That needs to tell `import a.b`, whose last component must be a submodule, from `from a import b` and `import a.b as x`, whose last component may be an ordinary attribute. lazy_pending_submodules now records which of the two a pending name came from, and IMPORT_FROM retracts the claim for the `as` form before anything is resolved. The repro in pythongh-151208 was fixed by pythongh-155194; this covers the shapes that survived it -- a sibling or parent import rebinding the name, and a lazy submodule registered after its package is loaded.
1 parent 8542958 commit 2b0aa85

12 files changed

Lines changed: 166 additions & 13 deletions

File tree

‎Include/internal/pycore_import.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ extern _PyLazySubmoduleImportResult _PyImport_TryLoadLazySubmodule(
4949
extern PyObject * _PyImport_LazyImportModuleLevelObject(
5050
PyThreadState *tstate, PyObject *name, PyObject *builtins,
5151
PyObject *globals, PyObject *locals, PyObject *fromlist, int level);
52+
extern int _PyImport_UnmarkPendingSubmodule(
53+
PyThreadState *tstate, PyObject *parent, PyObject *child);
5254

5355

5456
#ifdef HAVE_DLOPEN

‎Lib/test/test_lazy_import/__init__.py‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
import io
44
import dis
5+
import importlib
56
import subprocess
67
import sys
78
import textwrap
@@ -2209,6 +2210,28 @@ def test_import_after_variable_wins(self):
22092210
]
22102211
self.assertIs(module_same_name_var_order2.bar, bar_mod)
22112212

2213+
# gh-151208: `import pkg.sub` imports the submodule even when the package
2214+
# binds its own `sub`, as the eager import does.
2215+
def check_shadowed_submodule(self, driver):
2216+
mod = importlib.import_module(f"test.test_lazy_import.data.{driver}")
2217+
pkg = mod.test.test_lazy_import.data.submodule_name_shadowed
2218+
self.assertIsInstance(pkg.sub, types.ModuleType)
2219+
self.assertIs(pkg.sub, sys.modules[
2220+
"test.test_lazy_import.data.submodule_name_shadowed.sub"
2221+
])
2222+
2223+
def test_shadowed_submodule_then_parent_import(self):
2224+
"""A later `import pkg` should not lose a pending `import pkg.sub`."""
2225+
self.check_shadowed_submodule("import_parent_after_submodule")
2226+
2227+
def test_shadowed_submodule_then_sibling_import(self):
2228+
"""A later `import pkg.other` should not lose it either."""
2229+
self.check_shadowed_submodule("import_sibling_after_submodule")
2230+
2231+
def test_shadowed_submodule_of_loaded_parent(self):
2232+
"""A pending submodule binds on a package that is already loaded."""
2233+
self.check_shadowed_submodule("lazy_submodule_of_loaded_parent")
2234+
22122235

22132236
class DeletedModuleReimportTests(unittest.TestCase):
22142237
"""Tests for reimporting after module deletion from sys.modules."""
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
lazy import test.test_lazy_import.data.submodule_name_shadowed.sub
2+
lazy import test.test_lazy_import.data.submodule_name_shadowed
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
lazy import test.test_lazy_import.data.submodule_name_shadowed.sub
2+
lazy import test.test_lazy_import.data.submodule_name_shadowed.other
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
import test.test_lazy_import.data.submodule_name_shadowed
2+
lazy import test.test_lazy_import.data.submodule_name_shadowed.sub
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
sub = "shadows the sub submodule until it is imported"

‎Lib/test/test_lazy_import/data/submodule_name_shadowed/other.py‎

Whitespace-only changes.

‎Lib/test/test_lazy_import/data/submodule_name_shadowed/sub.py‎

Whitespace-only changes.

‎Makefile.pre.in‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2690,6 +2690,7 @@ TESTSUBDIRS= idlelib/idle_test \
26902690
test/test_lazy_import/data/metasyntactic/waldo/fred \
26912691
test/test_lazy_import/data/module_same_name_var_order1 \
26922692
test/test_lazy_import/data/module_same_name_var_order2 \
2693+
test/test_lazy_import/data/submodule_name_shadowed \
26932694
test/test_lazy_import/data/versioned \
26942695
test/test_module \
26952696
test/test_multiprocessing_fork \
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Fix a lazy ``import pkg.sub`` never importing the submodule when
2+
``pkg/__init__.py`` binds its own attribute of that name. The attribute
3+
answered the read that was meant to trigger the import, where an eager
4+
``import pkg.sub`` replaces it with the submodule.

0 commit comments

Comments
 (0)