Skip to content

Comments

Simplify setup code#1195

Merged
dalcde merged 25 commits intopyodide:masterfrom
hoodmane:simplify-setup
Feb 10, 2021
Merged

Simplify setup code#1195
dalcde merged 25 commits intopyodide:masterfrom
hoodmane:simplify-setup

Conversation

@hoodmane
Copy link
Member

@hoodmane hoodmane commented Feb 5, 2021

Now that #1167 is merged, it's possible to produce the globals object from Python. I've been thinking about deleting runpython.c for a while now, and I think this is the right time. I made a new function runPythonSimple which basically just calls _PyRun_SimpleString. Then the setup code in runpython.c can be done much simpler in python using runPythonSimple.

I think it also makes it easier to understand the C code now that the pyodide module isn't imported until main() is finished.

@dalcde
Copy link
Contributor

dalcde commented Feb 9, 2021

Can you resolve conflicts again?

@dalcde
Copy link
Contributor

dalcde commented Feb 9, 2021

From the user's point of view, what is the difference between raw_globals and globals?

@hoodmane
Copy link
Member Author

hoodmane commented Feb 9, 2021

raw_globals is a private implementation detail of the bootstrapping system, so only globals should exist from the user POV. From a technical perspective, I need to make raw_globals from C code in order to make the bootstrap work. The bootstrap process after this PR looks like:

  1. Execute all of the init functions for the various C "submodules".
  2. Define raw_globals in main.c. This is done with as little code as possible, I just need the simplest possible way to get an appropriate PyProxy in order to get the system going.
  3. Define runPythonSimple in pyodide.js, this is a trivial wrapper around PyRun_SimpleString.
  4. Define a setup function in runPythonSimple that takes the pyodide_js module as an argument and adds the globals and pyodide_py fields to the module.
  5. Use raw_globals.get("setup_function") to extract the setup function, then call it with Module (i.e., pyodide_js) as an argument.
  6. runPythonSimple("del setup_function")

I think this is an easier to understand bootstrapping process than what we had before because this way we don't import pyodide_py until execution of main() is completed. There is still the complication that registerJsModule relies on pyodide_py and so the js module cannot be accessed when pyodide_py is first imported.

I think we should split _importutils into a separate package to simplify things. Then we can first import _importutils and use registerJsModule to register the js package, and then pyodide_py is free to make top level js imports. Once we do this, I think it will be possible to modify the rest of pyodide_py without any special knowledge of the bootstrapping process.

@dalcde
Copy link
Contributor

dalcde commented Feb 9, 2021 via email

@hoodmane
Copy link
Member Author

hoodmane commented Feb 9, 2021

Great question. There are two reasons: builtins lookup, and the possibility of providing a special wrapper around globals to support access like pyodide.globals.some_name rather than pyodide.globals.get("some_name") (see discussion with @phorward on #1175).

I think we should probably just delete the pyodide.globals.some_name access in favor of pyodide.globals.get("some_name").

However, I think it is a good thing that e.g., pyodide.globals.get("dict") returns builtins["dict"] because most users won't make a strong distinction between global variables and builtins (they are slightly different but very similar in behavior). Currently builtins access through pyodide.globals is supported by saying globals.update(builtins.__dict__). I think this should be removed and we should make globals a javascript object that first attempts lookup in raw_globals (i.e., the normal globals) and then falls back to lookup on builtins. This closely mirrors the normal Python name lookup process.

@dalcde dalcde merged commit f64bee4 into pyodide:master Feb 10, 2021
@hoodmane hoodmane deleted the simplify-setup branch February 10, 2021 23:43
@rth rth mentioned this pull request Feb 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants