Implementation of the data model for v3.8#107
Conversation
This change begins a shift to a "plain Java objects" approach to the data model. We provide PyType and Operations classes, the marker interfaces and a base Python object implementation. The elaborate Slots enum describes a system of handles for special functions used by the run-time. Other classes, although numerous, are present to support these and errors they raise. There is a unit test that exercises creation and lookup of each broad sub-class of Operations object.
Move these Jython 2 source files unmodified to the Jython 3 source tree, preparatory to conversion for Jython 3. (Accept the build fails.) As a rule, each committed state should build without error. As an exception, we commit pure file moves from 2 to 3 to provide an unambiguous pedigree to git (and code archaeologists).
The changes to convert PyLong from 2 to 3 are large. Formatting code in stringlib can be re-used with relatively minor changes. This change set also brings in new classes for exceptions, and stubs for certain core classes to satisfy dependencies. We preserve legacy annotations in PyLong as comments so that the git diff can align the old and new implementations of corresponding methods.
We will use Python to generate some of our code, and it has to be a 3.8. We adjust the Travis CI build to support that, and insert a gratuitous use of Python as a test.
bool provides a simple case of inheritance but it is rather a special one since Java Boolean is its canonical implementation. We extend the Operations formation test to bool and int.
Allow code to be wide, but keep comments at a readable width. This aids comparison with the VSJ original, while we need it.
We provide an exposure mechanism sufficient to populate the dictionary of the type object for any built-in, and from that the slots of Operations objects. We apply this to int and bool. We test that we can invoke a few unary special methods for int and bool.
The refactoring extracts common code between the unary and binary tests (and later ones to be developed). The testing is at the level of descriptors: we are not yet caching these as method handles in the Operations object, although much of the code is present.
This change adds the code to Slot.java that causes the slots of Operations objects to be filled with handles to implementation methods. The unary and binary slot wrapper tests are extended to invoke these slots and test the result against the same criteria as for the corresponding descriptor.
This change imports Java code generation in Python and applies it to the repetitive task of creating a version of each PyLong method specialised for the class 'self'. Implementations inherited from Jython 2, still present for now, are ignored where a class-specific duplicate exists.
These are evolved versions of the Jython 2 code, moved to PyLongMethods, and called from generated wrappers. These operations are not simply invocations of the Java counterparts but quite complex, so we add a specific test.
Slots are added to Slot and OPerations based on CPython slots and how Jython works. Many of these are unused, but where they match methods in the ported PyLong, the signature will be validated. Since documentation is a focus, this change also fixes the javadoc so it builds cleanly.
Many of the special methods were already in the generated PyLongMethods, but now we remove them absolutely from PyLong. We add tests that call a few of them, to prove this works. We implement __(r)pow__ and the __*shift__ methods as hand-crafted additions to PyLongMethods, rather than generating them, as they are irregular.
We reproduce on the new code the style of copyright notice found elsewhere in Jython, with the addition of "Licensed to PSF ...", requested in the contributor agreement we all signed. The copyright label is probably not necessary, but we follow precedent.
|
I think this is ready for inspection now. Obviously it is nowhere near a Python interpreter, but only illustrates an aspect of the proposed new implementation, namely how we would invoke special methods (slots) on built-in types. This, however, is fundamental. And not that easy to follow, I should think, so here is a little code-walk of the idea. The best way to appreciate what's going on may be to run in your IDE the unit test AbstractAPITest, then single step one of its cases under the debugger, let's say the test of For a more complicated case, consider debugging the test of The action depends on Jython finding |
| // Same configuiration all sub-projects | ||
| tasks.withType(JavaCompile) { | ||
| options.encoding = 'UTF-8' | ||
| options.debug = true |
There was a problem hiding this comment.
Perhaps you want to set --release 11 too?
--release release
Compiles against the public, supported and documented API for a specific VM version. Supported release targets are 6, 7, 8, 9, 10, and 11.
Note:
When using --release for a version of the Java Platform that supports modules, you can’t use --add-modules to access internal JDK modules, nor can you use --add-exports to access internal JDK APIs in the modules.
There was a problem hiding this comment.
That seems a useful discipline to accept. The magic is:
tasks.withType(JavaCompile) {
// Use only public API
options.compilerArgs.addAll(['--release', '11'])
...
}
I'll slip it into main later.
|
Everyone seems content with this, so I'll rebase-merge, in this case keeping the commits as distinct, rather than squash. I expect this to "fast-forward", but the big green button always feels like a leap into the unknown. (I intend to take @thetric's suggestion as a separate change.) |
|
Process note: GitHub has not created a needless merge commit, but it did needlessly reproduce all the commits as new objects. :( Harmless in the long run, but I believe I could have avoided this at the prompt. |
3 participants
This is a proposed implementation for objects in the core of Jython 3. It is the model I talked about at the on-line Python Language Summit 2021, applied to the single Python type
int. It is foundational for, but does not include, the use of dynamic call sites for performance.It is draft as I'd like to push more onto it before it gets into the main repo. I thought it good to have it out there as a PR for those that want to follow along.
It is somewhat unsatisfactory as a PR because it includes a lot of code that has no purpose without parts I intend to leave to later PRs. I'll add more tests, I think, and maybe the exposure mechanism, but avoid the trap of piling everything in one massive PR.