Signum supposed to be used after base64 decoding:
https://github.com/auth0/jwks-rsa-java/blob/master/src/main/java/com/auth0/jwk/Jwk.java#L196-L197

Sample:
ECPoint ecPoint = new ECPoint(new BigInteger(1, Base64.getUrlDecoder().decode(stringValue("x"))),
new BigInteger(1, Base64.getUrlDecoder().decode(stringValue("y"))));

Similar approach is already used in lines 183-184 for RSA Algorithm.

Issue causes invalid pubKey decoding resulting in signature validation failure.