(not done yet: making it respect scalac.args.optimise)

hello permgen our old friend you've come to talk to us again

sbt.ForkMain$ForkError: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error running /home/jenkins/workspace/scala-2.11.x-validate-test/test/files/scalacheck/Ctrie.scala
Caused by: sbt.ForkMain$ForkError: java.lang.OutOfMemoryError: PermGen space

if I run test/partest test/files/scalacheck/Ctrie.scala by itself, locally, it passes even with -XX:MaxPermSize=48M, so it seems this isn't an unusually PermGen hungry test. which has me worried scala-partest-interface might be doing something PermGen-unfriendly, perhaps holding on to classloader references between tests.

increasing PermGen from 128M to 192M didn't help — and failed on the exact same test.

note also # 111/117 passed, 6 failed in jvm ("compilation failed" errors)

sbt/sbt#1223 might offer some clues. SBT forces GC and finalization periodically if the number of objects pending finalization exceeds a threshold. It appears that this is only done after a task has executed, so it probably won't help if a single task like tests/it:test internally leaks.

The permgen failure also happens when running locally on Mac OS X. It sat a long time before failing; sample stack trace not long before failing: https://gist.github.com/SethTisue/8db45ef617a0542057b2

Strange, I can run partest locally from sbt without problems.

sbt/sbt#1223 might offer some clues

I don't think so, because partest runs in a forked JVM.

Strange, I can run partest locally from sbt without problems

@szeiger on Java 6?

Hah, good point, I probably only ran it on Java 8.

as I study a log of the old test/validate script a little closer, I see that ant was invoking partest multiple times, once per category of test. whereas in sbt we're doing one single giant partest invocation. so it's actually not surprising this might cause a PermGen regression, and I don't think there's any need to investigate the details.

I'll just change it to match the old setup. the easiest way, as @retronym suggests, is simply not to invoke partest through sbt at all. at some point, we might like to change test/it:partest to do the multiple invocations, but for now, we can just bypass sbt-partest-interface altogether. this still accomplishes our main goal which is to get ant out of the picture.

@SethTisue you can run individual categories from sbt, e.g. test/it:testOnly -- --run. I hope the syntax is right. I'm on Windows right now and I just noticed that partest in sbt tries to run pull-binary-libs for some reason. I need to look into that.

you can run individual categories from sbt

good to know. ultimately, we should switch to that.

it didn't fail the job, but:

Archiving artifacts
WARN: No artifacts found that match the file pattern "build/junit/TEST-*,build/osgi/TEST-*,hs_err_*.log". Configuration error?
WARN: ‘build/junit/TEST-*’ doesn’t match anything: ‘build’ exists but not ‘build/junit/TEST-*’

sigh, test/partest actually runs ant -q test.suite.init, so we're not cutting ant out of the picture by calling test/partest unless that's changed, and it also then reads partest.properties to construct a classpath, and that classpath uses the ant build directory, not the sbt build directory. so this needs more work.

and it also then reads partest.properties to construct a classpath

The SBT build now generates this file, albeit under ./build-sbt.

See 2a57f82

We'd still need to modify the partest script to avoid calling ant, perhaps when provided with --no-ant.

That explains why it fails on Windows. I don't think partest or sbt-partest-interface should call any script at all. All that suite.init does is to build the classpath for partest in build/pack/partest.properties, which triggers a full ant build and pulling the libraries with the shell script, because all that stuff goes on the classpath. I wonder why I haven't noticed this before. sbt can build a correct partest.properties (but it goes into build-sbt/quick) which works for running partest from the command line. We should pass that to partest from sbt instead of calling ant.

It looks like the problem with partest in sbt is really just

I tested this on a Windows system that doesn't even have ant installed. This includes running partest from the shell (MSYS bash) after sbt dist/mkQuick and cp build-sbt/quick/partest.properties build/pack/. No ant and no JARs in lib required.

MiMa failure:

[info] Checking backward binary compatibility
[error] (run-main-1) java.lang.ArrayIndexOutOfBoundsException: 1477
[error] (run-main-0) java.lang.ArrayIndexOutOfBoundsException: 1477
java.lang.ArrayIndexOutOfBoundsException: 1477
    at com.typesafe.tools.mima.core.BufferReader.nextByte(BufferReader.scala:33)
    at com.typesafe.tools.mima.core.BufferReader.nextInt(BufferReader.scala:53)
    at com.typesafe.tools.mima.core.ClassfileParser$$anonfun$skipAttributes$1.apply$mcVI$sp(ClassfileParser.scala:269)

doesn't happen when I run locally. perhaps the 2.11.0 jar it's trying to use isn't a real jar, perhaps because retrieving it failed because validation stuff is running with special resolver settings, that kind of thing. I'll look into it (but not right away).

MiMa failure: [...]

turned out to be transient, so I think the MiMa part of this is done (& I can move on to the other remaining parts)

Closing in favor of #5190