Hello! I work on Datadog's ddtrace gem and noticed this issue on our CI today.
The latest msgpack version, when run with the Java 8 VM breaks:
$ docker run jruby:9.3-jre8 bash -c "gem install msgpack; ruby -rmsgpack -e 'puts RUBY_DESCRIPTION; [1,2,3].to_msgpack'"
/opt/jruby/lib/ruby/stdlib/socket.rb:4: warning: already initialized constant Socket::Constants::AF_INET6
/opt/jruby/lib/ruby/stdlib/socket.rb:4: warning: already initialized constant Socket::Constants::IPPROTO_ICMPV6
/opt/jruby/lib/ruby/stdlib/socket.rb:4: warning: already initialized constant Socket::Constants::IPPROTO_IPV6
/opt/jruby/lib/ruby/stdlib/socket.rb:4: warning: already initialized constant Socket::Constants::PF_INET6
/opt/jruby/lib/ruby/stdlib/socket.rb:4: warning: already initialized constant Socket::Constants::SOL_IPV6
Successfully installed msgpack-1.4.3-java
1 gem installed
jruby 9.3.3.0 (2.6.8) 2022-01-19 b26de1f5c5 OpenJDK 64-Bit Server VM 25.312-b07 on 1.8.0_312-b07 +jit [linux-x86_64]
Unhandled Java exception: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
readRubyString at org/msgpack/jruby/Encoder.java:70
encode at org/msgpack/jruby/Encoder.java:81
write at org/msgpack/jruby/Packer.java:128
call at org/msgpack/jruby/Packer$INVOKER$i$1$0$write.gen:-1
cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:372
call at org/jruby/runtime/callsite/CachingCallSite.java:175
processCall at org/jruby/ir/interpreter/InterpreterEngine.java:316
Note that I've used JRuby 9.3 above, but it also happens on 9.2, since the issue is with the version of the Java VM:
$ docker run jruby:9.2-jre8 bash -c "gem install msgpack; ruby -rmsgpack -e 'puts RUBY_DESCRIPTION; [1,2,3].to_msgpack'"
Successfully installed msgpack-1.4.3-java
1 gem installed
jruby 9.2.20.1 (2.5.8) 2021-11-30 2a2962fbd1 OpenJDK 64-Bit Server VM 25.312-b07 on 1.8.0_312-b07 +jit [linux-x86_64]
Unhandled Java exception: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
readRubyString at org/msgpack/jruby/Encoder.java:70
encode at org/msgpack/jruby/Encoder.java:81
write at org/msgpack/jruby/Packer.java:128
call at org/msgpack/jruby/Packer$INVOKER$i$1$0$write.gen:-1
cacheAndCall at org/jruby/runtime/callsite/CachingCallSite.java:376
call at org/jruby/runtime/callsite/CachingCallSite.java:175
processCall at org/jruby/ir/interpreter/InterpreterEngine.java:316
This test works correctly if instead of the jruby:9.2-jre8 and jruby:9.3-jre8 we switch to jruby:9.2-jre11 and jruby:9.3-jre11.
My analysis of the issue is as follows:
In Encoder.java the code calls buffer.clear():
The buffer here refers to a java.nio.ByteBuffer. In Java 8, ByteBuffer::clear is actually inherited from its parent class:
Looking at its parent class, the signature for clear is:
and thus in ByteBuffer the signature for calling this method using Java bytecode is
java.nio.ByteBuffer.clear()Ljava/nio/Buffer;
...whereas msgpack seems to be looking for a
java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
The latter method was actually added in Java 9:
Notice how it says that it overrides the method clear, and the return is now ByteBuffer, not just Buffer. (Having a subclass override a return with a more specific type is allowed in Java).
Now it is true that msgpack is compiled with source and target version 1.8 (aka Java 8):
|
ext.source_version = '1.8' |
|
ext.target_version = '1.8' |
But I'm betting that the release was not compiled on Java 8, and that we're running into this known limitation (from here):
If you do not specify the correct version of bootstrap classes, the compiler will use the old language rules (in this example, it will use version 1.6 of the Java programming language) combined with the new bootstrap classes, which can result in class files that do not work on the older platform (in this case, Java SE 6) because reference to non-existent methods can get included.
TL;DR it's not enough to use source and target as Java 1.8, you actually need to be running Java 8 OR feed the compiler with a Java 8 rt.jar file.
Hello! I work on Datadog's ddtrace gem and noticed this issue on our CI today.
The latest msgpack version, when run with the Java 8 VM breaks:
Note that I've used JRuby 9.3 above, but it also happens on 9.2, since the issue is with the version of the Java VM:
This test works correctly if instead of the
jruby:9.2-jre8andjruby:9.3-jre8we switch tojruby:9.2-jre11andjruby:9.3-jre11.My analysis of the issue is as follows:
In
Encoder.javathe code callsbuffer.clear():msgpack-ruby/ext/java/org/msgpack/jruby/Encoder.java
Line 70 in f706c95
The
bufferhere refers to ajava.nio.ByteBuffer. In Java 8,ByteBuffer::clearis actually inherited from its parent class:Looking at its parent class, the signature for
clearis:and thus in
ByteBufferthe signature for calling this method using Java bytecode isjava.nio.ByteBuffer.clear()Ljava/nio/Buffer;...whereas msgpack seems to be looking for a
java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;The latter method was actually added in Java 9:
Notice how it says that it overrides the method
clear, and the return is nowByteBuffer, not justBuffer. (Having a subclass override a return with a more specific type is allowed in Java).Now it is true that msgpack is compiled with source and target version 1.8 (aka Java 8):
msgpack-ruby/Rakefile
Lines 37 to 38 in 1af1773
But I'm betting that the release was not compiled on Java 8, and that we're running into this known limitation (from here):
TL;DR it's not enough to use source and target as Java 1.8, you actually need to be running Java 8 OR feed the compiler with a Java 8
rt.jarfile.