Skip to content

Conversation

@vstinner
Copy link
Member

@vstinner vstinner commented Jun 19, 2023

Explain in What's New in Python 3.12 how to port existing code using the removed imp to the importlib module.


📚 Documentation preview 📚: https://cpython-previews--105905.org.readthedocs.build/

@vstinner
Copy link
Member Author

Replacing load_dynamic() is non-trivial:

def load_dynamic(name, filename):
    loader = importlib.machinery.ExtensionFileLoader(name, filename)
    spec = importlib.util.spec_from_loader(name, loader)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module.__name__] = module
    loader.exec_module(module)
    return module

If you omit loader.exec_module(), you get an uninitialized extension which lead to funny crashes :-)

@vstinner
Copy link
Member Author

Replace init_builtin():

def init_builtin(name):
    spec = importlib.machinery.BuiltinImporter.find_spec(name)
    if spec is None:
        raise ImportError(f'no built-in module named {name!r}')
    return importlib.util.module_from_spec(spec)

@vstinner
Copy link
Member Author

For removed imp.load_source(), I proposed adding importlib.util.load_source_path(): PR #105755.

@vstinner
Copy link
Member Author

Replace imp.load_package():

import sys
import os.path
import importlib

def load_package(name, path):
    old_path = list(sys.path)
    try:
        sys.path.insert(0, os.path.dirname(path))
        return importlib.import_module(name)
    finally:
        sys.path.clear()
        sys.path.extend(old_path)

I'm not sure about this one. It uses high-level API to call indirectly PathFinder and FileFinder API. I would prefer to not expose them since I consider them as the internal API.

@vstinner
Copy link
Member Author

For removed imp.load_source(), I proposed adding importlib.util.load_source_path(): PR #105755.

I suppose that it can be used to replace imp.load_cached().

Otherwise, here is a recipe to replace imp.load_cached():

import importlib.util
import sys

def load_compiled(name, filename):
    spec = importlib.util.spec_from_file_location(name, filename)
    module = importlib.util.module_from_spec(spec)
    sys.modules[module.__name__] = module
    spec.loader.exec_module(module)
    return module

Test:

import os.path

def create_pyc():
    name = 'script'
    filename = f'/tmp/{name}.py'
    with open(filename, "w") as fp:
        print("print('hello')", file=fp)

    sys.path.insert(0, os.path.dirname(filename))
    mod = importlib.import_module(name)

    del sys.path[0]
    del sys.modules[name]
    os.unlink(filename)
    mod = None

    return importlib.util.cache_from_source(filename)

print("create PYC")
pyc = create_pyc()

print(f"load PYC: {pyc}")
mod = load_compiled('script', pyc)
print(mod)
print(mod.__file__)
print(mod.__cached__)
os.unlink(pyc)

@vstinner
Copy link
Member Author

My recipes always create a new module: they don't try to get a cached module from sys.modules, whereas the imp module respected sys.modules cache.

Explain in What's New in Python 3.12 how to port existing code using
the removed imp to the importlib module.
@vstinner vstinner merged commit 7a56a41 into python:main Jun 19, 2023
@vstinner vstinner deleted the port_imp branch June 19, 2023 14:13
@vstinner vstinner added the needs backport to 3.12 only security fixes label Jun 20, 2023
@miss-islington
Copy link
Contributor

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-105952 is a backport of this pull request to the 3.12 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.12 only security fixes label Jun 20, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 20, 2023
vstinner added a commit that referenced this pull request Jun 20, 2023
… (#105952)

gh-104212: Explain how to port imp code to importlib (GH-105905)
(cherry picked from commit 7a56a41)

Co-authored-by: Victor Stinner <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation in the Doc dir skip news

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants