-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Ensure ClassBTypes constructed from symbol and classfile are identical #5096
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
JUnit 4 does not support running `@Test` methods defined as default methods in parent interfaces. JUnit 5 will, but is not yet available. Currently scalac emits a forwarder to every trait method inherited by a class, so tests are correctly executed. The fix for SD-98 will change this.
|
I did not really read/understand the changes, but I'm guessing this could cause the opposite "problem" in Scala.js: that classes (symbols) read from classfiles do have the synthetic parents, but that symbols being compiled do not have the synthetic parents at the time Scala.js runs. The Scala.js IR does not have the restriction that super calls may only call directly inherited interfaces, so the Scala.js back-end would not add synthetic parents to fix the problem. I'm not exactly sure whether what I'm saying makes any sense, but maybe it will trigger some thought. |
Nothing has changed for symbols and frontend types. the additional interfaces only show up in we have two ways of filling up the details of a |
OK then my concern is moot. Scala.js does not use |
In most cases when a class inherits a concrete method from a trait we
don't need to generate a forwarder to the default method in the class.
t5148 is moved to pos as it compiles without error now. the error
message ("missing or invalid dependency") is still tested by t6440b.
It tests an internal debugging tool which does not appear to work as intented. If anyone can compile and run that test and get an output that looks like the check file, I'd be interested to know. Origins does not seem to support the kind of stack traces that scalac currently emits.
748c5e9 to
f36aa1c
Compare
|
here's an observation while working on this PR: there's no good way to generate a valid tree for a static method defined in scala. for comparison, the selection of a java-defined static method assume we have a static method in a scala class:
in any case, the backend will identify the selection of a static method and not generate any bytecode for the these are not blockers, just a little weird. we should keep them in mind while adding more static methods (https://issues.scala-lang.org/browse/SI-9390) |
057309d to
6fd8275
Compare
| if (sym == definitions.Array_clone) { | ||
| // Special-case Array.clone, introduced in 36ef60e. The goal is to generate this | ||
| // call as "[I.clone" instead of "java/lang/Object.clone" (which would also work). | ||
| // Not sure why this is useful, maybe for performance. The commit message says |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that's because Object.clone() is protected. I guess trying to call it would be an illegal access on the JVM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested without the special case:
object Test {
def foo(a: Array[Int]) = a.clone()
def main(args: Array[String]): Unit = {
println(foo(Array(1,2,3)).toList)
}
}
gives
public final class Test$ {
public foo([I)[I
ALOAD 1
INVOKEVIRTUAL java/lang/Object.clone ()Ljava/lang/Object;
CHECKCAST [I
ARETURN
}
and it runs fine. but i'd have to check the spec of protected again..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found this part of the JVM spec hard to follow. clone is overriden as public in each of the Array classes (which also implement Clonable). Seems like its fine to refer the the protected method in Object in the invokevirtual in this case.
Toying around with this via reflection:
scala> java.lang.invoke.MethodHandles.publicLookup.findVirtual(classOf[Object], "clone", java.lang.invoke.MethodType.methodType(classOf[Object]))
java.lang.IllegalAccessException: member is protected: java.lang.Object.clone()Object/invokeVirtual, from java.lang.Object/public
at java.lang.invoke.MemberName.makeAccessException(MemberName.java:852)
scala> def lookupOf(cls: Class[_]) = { import java.lang.invoke._; val constructor = classOf[MethodHandles.Lookup].getDeclaredConstructors.head; constructor.setAccessible(true); constructor.newInstance(cls).asInstanceOf[MethodHandles.Lookup] }
lookupOf: (cls: Class[_])java.lang.invoke.MethodHandles.Lookup
scala> lookupOf(classOf[Array[Int]]).findVirtual(classOf[Object], "clone", java.lang.invoke.MethodType.methodType(classOf[Object]))
res4: java.lang.invoke.MethodHandle = MethodHandle(int[])Object
Perhaps related:
https://bugs.openjdk.java.net/browse/JDK-4750641
https://bugs.openjdk.java.net/browse/JDK-4962591
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like OpenJDK has a special case in method resolution to allow this. Probably best not to rely on it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Checked again the JVM spec
Access Control: protected method R, declared in C, is accessible to a class or interface D if
- D is a subclass of C, and
- R must contain a symbolic reference to a class T, such that T is either a subclass of D, a superclass of D, or D itself
"symbolic reference to class T" means the receiver class in the invocation instruction.
But there's more: "This discussion omits a related restriction [...] is checked as part of the verification process (§4.10.1.8); it is not part of link-time access control."
4.10.1.8. Type Checking for protected Members encodes that a protected method C.m can be accessed in a subclass D only if the receiver object's type conforms to D. This corresponds to the Java spec 6.6.2.1.
Assume we have
class C { override def clone(): Object = "hi" }We cannot emit def f(c: C) = c.clone() as Object.clone(). he JVM would perform the following checks:
- Method Resolution (5.4.3.3) finds
Object.clone - Access Control (5.4.4) is checked at the end of resolution and succeeds (every class and interface is a subclass of
Object, soObject.clonecan always passes) - Verification (4.10.1.8) will fail the
passesProtectedCheck
I tested this with a compiler hack, we get java.lang.VerifyError indeed.
So the reason Object.clone works for arrays is really the special cases you point to above. Given that javac also emits [I.clone we better stick to that.
Finally, to make the discussion complete, here's the example with traits:
trait T { override def clone(): Object = "hu" }
trait U extends T
class K extends U
object T1 { def f1(u: U) = u.clone() }
We have to emit u.clone as T.clone. Using U.clone gives NoSuchMethodError: U.clone. Looking at 5.4.3.4. Interface Method Resolution I actually don't see why this is the case, it might be a bug in the JVM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To stress that point ("it might be a bug in the JVM"), I can get javac to produce bytecode that crashes with a NoSuchMethodError:
public class A {
public interface I1 {
default Object clone() { return "hi"; }
}
public interface I2 extends I1 { }
public static class K implements I2 {
public Object clone() { return "Ka"; }
}
public static void f(I2 i) { System.out.println(i.clone()); }
public static void main(String[] args) {
f(new K());
}
}$ javac A.java
$ java A
Exception in thread "main" java.lang.NoSuchMethodError: A$I2.clone()Ljava/lang/Object;
at A.f(A.java:12)
at A.main(A.java:15)
I remember noticing the same problem once before when working in Another possibility would be be to use a null constant for the qualifier. |
FYI, currently the Scala.js backend expects this. It does not do anything special to identify static methods. It compiles them as calls to methods on the companion object, just as the tree says. That's what it's supposed to do, because Java statics end up being re implemented in Scala as methods on the companion object. |
|
@sjrd Here's a case where Scala 2.11 currently emits a static method within a class. This is done as a special case in lambdalift to avoid a verify error. Might be interesting to look at this from the ScalaJS perspective. |
|
@retronym That's ... interesting ^^ I never saw that pop up. It turns out it works fine because the method is also emitted as non-static, and that is not a problem for the Scala.js IR: |
| lineNumber(tree) | ||
| fieldStore(lhs.symbol) | ||
| // receiverClass is used in the bytecode to access the field. using sym.owner may lead to IllegalAccessError, SI-4283 | ||
| val receiverClass = qual.tpe.typeSymbol |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks consistent with javac.
public class Test {
private static class A {
int i;
static int j;
void m() {};
static void n() {};
}
static class B extends A {};
public static void main(String[] args) {
B b = new B();
C.test(b);
assert(b.i == 1);
}
}
class C {
public static void test(Test.B b) {
b.i = b.i + 1;
b.j = b.j + 1;
b.m();
b.n();
}
} public static void test(Test$B);
Code:
0: aload_0
1: aload_0
2: getfield #2 // Field Test$B.i:I
5: iconst_1
6: iadd
7: putfield #2 // Field Test$B.i:I
10: aload_0
11: pop
12: aload_0
13: pop
14: getstatic #3 // Field Test$B.j:I
17: iconst_1
18: iadd
19: putstatic #3 // Field Test$B.j:I
22: aload_0
23: invokevirtual #4 // Method Test$B.m:()V
26: aload_0
27: pop
28: invokestatic #5 // Method Test$B.n:()V
31: return
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Scala doesn't allow static references via an instance or a subclass qualifier. I just noticed that this makes it impossible to express some static field/method selections.
class D {
def test(b: Test.B) {
b.i = b.i + 1;
b.j = b.j + 1; // j not a member
Test.B.j = Test.B.j + 1; // j not a member
Test.A.j = Test.A.j + 1; // A is not accessible
b.m();
b.n(); // n is not a member
Test.B.n(); // n is not a member
Test.A.n(); // A is not accessible
}
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For static access, the qual.tpe.typeSymbol will be the (fictional) module class representing the static perspective of the Java class. Looks like this is correctly handled by def internalName, so this looks correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Scala doesn't allow static references via an instance or a subclass qualifier
I didn't know that was possible in Java..!
|
We may need a related change in public class Test {
private static class A {
int i;
}
static class B extends A {
};class D {
def test(b: Other with Test.B) {
b.i = b.i + 1;
}
}
class Other// scalac-2.11.8 -Xprint:posterasure sandbox/test.scala
[[syntax trees at end of posterasure]] // test.scala
package <empty> {
class D extends Object {
def test(b: Other): Unit = b.$asInstanceOf[Test.A]().i = b.$asInstanceOf[Test.A]().i.+(1)
};
}Here, we have cast the qualifier to the owner of the field. I expect this would be pretty hard to fix, though, and it is something of a corner case so we might just have to leave it. |
|
Good observation..! I managed to turn this into a test: |
|
Echoing the earlier discussion about Jason:
The special case basically considers Lukas:
Lukas:
|
|
@retronym I updated the comments here and here and filed https://issues.scala-lang.org/browse/SI-9761, so this PR should be ready now. |
|
LGTM |
The code was patched many times in the history and became a bit scattered. When emitting a virtual call, the receiver in the bytecode cannot just be the method's owner (the class in which it is declared), because that class may not be accessible at the callsite. Instead we use the type of the receiver. This was basically done to fix - aladdin bug 455 (9954eaf) - SI-1430 (0bea2ab) - basically the same bug, slightly different - SI-4283 (8707c9e) - the same for field reads In this patch we extend the fix to field writes, and clean up the code. This patch basically reverts 6eb55d4, the fix for SI-4560, which was rather a workaround than a fix. The underlying problem was that in some cases, in a method invocation `foo.bar()`, the method `bar` was not actually a member of `foo.tpe`, causing a NoSuchMethodErrors. The issue was related to trait implementation classes. The idea of the fix was to check, at code-gen time, `foo.tpe.member("bar")`, and if that returns `NoSymbol`, use `barSym.owner`. With the new trait encoding the underlying problem seems to be fixed - all tests still pass (run/t4560.scala and run/t4560b.scala).
A super call (invokespecial) to a default method T.m is only allowed if
the interface T is a direct parent of the class. Super calls are
introduced for example in Mixin when generating forwarder methods:
trait T { override def clone(): Object = "hi" }
trait U extends T
class C extends U
The class C gets a forwarder that invokes T.clone(). During code
generation the interface T is added as direct parent to class C. Note
that T is not a (direct) parent in the frontend type of class C.
This commit stores interfaces that are added to a class during code
generation in the InlineInfo classfile attribute. This allows filtering
the interface list when constructing a ClassBType from a classfile.
For some reason this was not the case, leading to spurious inliner warnings (no inline info found for method O$lzycompute).
No description provided.