Use sbt for PR validation [ci: last-only] #5190

szeiger · 2016-05-25T15:40:58Z

Picking up the work on #4933

szeiger · 2016-05-25T17:31:48Z

Ah, we still need pull-binary-libs call for partest to fetch some JARs. The larger jsoup JAR could probably be resolved from Maven Central and the bunch of small JARs added directly to the git repo.

But this doesn't explain the test failure. The pull-binary-libs call is still there and you can clearly see from the log that it resolves the files, but none of them end up on the scalac classpath for partest. I was unable to reproduce this issue locally.

retronym · 2016-05-26T03:26:10Z

@szeiger Related discussion about pull-binary-libs: https://issues.scala-lang.org/browse/SI-8379

retronym · 2016-05-26T03:28:11Z

bunch of small JARs added directly to the git repo.

We could also rewrite the tests to generate the necessary bytecode programmatically. We can use ASMifier to reify the ASM code that would create a given class, and then whittle that down the miminum needed to provoke the original bug.

Of course, we might just be able to include Java source files and use the separate compilation test case feature of partest JavaSource_1.java / ScalaClient_2.scala. Some of the JARs might exists because we couldn't bank on a recent enough Java compiler being available, perhaps when we supported, but did not require, Java 1.5.

szeiger · 2016-05-30T15:57:05Z

It looks like we have our first successful PR validation build on sbt. Remaining issues:

Run scaladoc tests from sbt
Port OSGi tests to sbt
Put the build artifacts in the same location as the ant build or change the Jenkins configuration:

Archiving artifacts
WARN: No artifacts found that match the file pattern "build/junit/TEST-*,build/osgi/TEST-*,hs_err_*.log". Configuration error?
WARN: ‘build/junit/TEST-*’ doesn’t match anything: ‘build’ exists but not ‘build/junit/TEST-*’

The following are non-issues (at least for this PR) IMHO:

Respect scalac.args.optimise - The PR validation build is always with optimization. If we need other builds without, they get different setup commands (in project/ScriptCommands.scala) where optimization can be left off.
Do not require pull-binary-libs for PR validation (or anything else) - We should do this but it's not a requirement for switching from ant to sbt, so not in scope for this PR.

ghost · 2016-05-30T16:12:42Z

re scaladoc. We have issues generating scaladoc from sbt in cats. I know the issue/solution but have not PR'd anything- ref chat https://gitter.im/sbt/sbt?at=574b4bde10f0fed86f49499c

In case you hit any issues before I PR anything, please shout.

szeiger · 2016-05-30T17:15:04Z

@inthenow Scaladoc generation has been working in the Scala build for months. The remaining issue here is about running tests on the scaladoc tool (using partest from sbt)

szeiger · 2016-06-03T13:00:55Z

Marking "Put the build artifacts in the same location as the ant build or change the Jenkins configuration" as solved. Some build artifacts were produced by the ant build and then archived by Jenkins because they contain additional details about build errors. The sbt build logs everything to the console instead, so these files should not be necessary anymore. The warning can be safely ignored and the Jenkins configuration changed after this PR has been merged.

szeiger · 2016-06-03T16:13:37Z

Ready for review.

adriaanm · 2016-06-03T23:47:11Z

build.sbt

 }

+// This project provides the STARR scalaInstance for bootstrapping
+lazy val bootstrap = (project in file("target/bootstrap")).settings(


i wonder about the various ways to create Files that we see in build.sbt:

file("target/...")

file(".") / "target" / ...

(baseDirectory in ThisBuild).value / "target" / ...

buildDirectory.value / "quick" / ...

does it matter, should we unify / clean up a bit?

We should be able to get rid of many of those when we move to a more standard sbt layout after ant is gone. (The first 3 all do the same, the last one is different)

here: if you tell me we're fine for now then that's fine with me :)

adriaanm · 2016-06-04T00:32:34Z

I would really like to reduce the additional noise in the output. Maybe suppress [info]-level output? I know it's annoying to fix, but if we postpone it I'm afraid we'll just end up living with it.

retronym · 2016-06-04T00:59:47Z

I just tried to use the native SBT import in IntelliJ to see if the changes in this PR had been enough to get it working. (It wasn't working before because of our custom scalaInstance.)

It does successfully complete, but ends up with the wrong directory mappings:

Not a blocker for this PR, just a pity.

retronym · 2016-06-04T01:11:51Z

It does successfully complete, but ends up with the wrong directory mappings:

Oh, that's a side effect of a change I made previously:

  // When we fork subprocesses, use the base directory as the working directory.
  // This enables `sbt> partest test/files/run/t1.scala` or `sbt> scalac sandbox/test.scala`
  baseDirectory in Compile := (baseDirectory in ThisBuild).value,
  baseDirectory in Test := (baseDirectory in ThisBuild).value,

IntelliJ looks up compile:baseDirectory rather than baseDirectory to use as the content root of the module.

szeiger · 2016-06-06T09:38:36Z

@adriaanm What noise are you referring to specifically? The OSGi tests are noisy but I didn't bother trying to change that (not sure that I could - I only managed to turn some of that noise off in Slick's OSGi tests) because you don't normally run them manually and the console output replaces the JUnit XML logs from the ant build.

adriaanm · 2016-06-06T14:27:13Z

Most lines that start with [info]. For example, the ones about resolving .... The JUnit tests also output more than necessary.

lrytz · 2016-06-07T08:56:59Z

build.sbt

-          def annotationName = "partest"
-        }, true, Array())
-     )
+    testOptions in IntegrationTest += Tests.Argument("-Dpartest.scalac_opts=" + (scalacOptions in Compile).value.mkString(" ")),


it looks like space separation might not work if there are multiple scalac options

-Dpartest.scalac_opts=-optimize -deprecation

maybe surround the options by single quotes? just guessing..

No, there's no parsing of quoted arguments happening anywhere. This is a single Argument for partest which does a .split(' ') on the RHS of the -D def to get the scalac args. There is currently no way use spaces inside of scalac args but that's a limitation of partest (and is also present when launching it from ant or the command line).

OK, thanks!

szeiger · 2016-06-08T16:09:59Z

/rebuild

lrytz · 2016-06-08T18:16:18Z

Caused by: sbt.ForkMain$ForkError: java.lang.RuntimeException: 'instrumented.jar' not found in '/home/jenkins/workspace/scala-2.11.x-validate-test/test/files/speclib'.

retronym · 2016-06-09T07:37:27Z

Most lines that start with [info]. For example, the ones about resolving .... The JUnit tests also output more than necessary.

@adriaanm I turned that off in https://github.com/scala/scala/pull/5152/files#diff-991b9473f196f6364820f0c3652eb3faR32 . Perhaps you are on a branch doesn't have that change it its hiatory yet?

szeiger · 2016-06-09T11:11:31Z

sigh The reference to speclib/intrumented.jar is hardcoded in partest itself

szeiger · 2016-06-09T16:38:03Z

@retronym I've just pushed another commit with a workaround for your scalacheck bootstrapping problem (provided that it passes CI and doesn't cause new problems).

adriaanm · 2016-06-10T08:17:07Z

Perhaps you are on a branch doesn't have that change it its hiatory yet?

I was looking at the output of this PR, so I guess this that'll take care of itself when we merge.

szeiger · 2016-06-10T09:43:28Z

No, it's in the history and I see the code. It may be scoped too narrowly to catch everything. I'll investigate. (BTW, noise has already been considerably reduced by running everything from a single sbt invocation; now you only get the resolver spam once at the beginning)

szeiger · 2016-06-13T10:26:09Z

Everything should be fixed now.

adriaanm · 2016-06-13T10:27:08Z

Fix all the things!! :party:

szeiger · 2016-06-14T10:40:06Z

Squashed, rebased and switched to partest 1.0.15

dwijnand · 2016-06-14T10:50:21Z

project/MiMa.scala

+// It would be nice to use sbt-mima-plugin here, but the plugin is missing
+// at least two features we need:
+// * ability to run MiMa twice, swapping `curr` and `prev`, to detect
+//   both forwards and backwards incompatibilities


I think this isn't true as of 0.1.9: lightbend-labs/mima@2844ffa

- Support directories in `-doc-external-doc`: It is documented as accepting a “classpath_entry_path” for the keys but this only worked for JARs and not for individual class files. When checking for external-doc mappings for a Symbol, we now find the root directory relative to a class file instead of using the full class file path. The corresponding tests for SI-191 and SI8557 are also fixed to support individual class files instead of JARs in partest. This is required for the sbt build which runs partest on “quick” instead of “pack”. - Fix version and repository handling for bootstrapping. The bootstrap `scalaInstance` can now be resolved from any repository added to the project (not just the bootstrap repositories) by using a different workaround for sbt/sbt#1872. - Workaround for sbt/sbt#2640 (putting the wrong `scalaInstance` on partest’s classpath). The required `ScalaInstance` constructor is deprecated, so we have to disable deprecation warnings and fatal warnings until there is a better fix. - Add MiMa to the sbt build (port of the old `test.bc` ant task). The sbt-mima plugin doesn’t have all the features we need, so we do it manually in a similar way to what the plugin does. Checks are done in both directions for the `library` and `compiler` projects. The base version has to be set in `build.sbt`. When set to `None`, MiMa checks are skipped. MiMa checks are run sequentially to avoid spurious errors (see lightbend-labs/mima#115). - Port the OSGi tests to the sbt build. The set of JARs that gets copied into build/osgi as bundles is a bit different from the ant build. We omit the source JARs but add additional modules that are part of the Scala distribution, which seems more correct. - Get rid up `pull-binary-libs.sh` for the sbt build. Add artifacts are resolved from the special bootstrap repository through Ivy. The special `code.jar` and `instrumented.jar` artifacts are copied to the location where partest expects them (because these paths are hardcoded in partest). Other extra JARs for partest in `test/files/lib` are referenced directly from the Ivy cache. - Move common settings that should be available with unqualified names in local `.sbt` files and on the command line into an auto-plugin. - Add an `antStyle` setting to sbt to allow users to easily enable ant-style incremental compilation instead of sbt’s standard name hashing with `set antStyle := true`. - Disable verbose `info`-level logging during sbt startup for both, `validate/test` and `validate/publish-core` jobs. Update logging is no longer disabled when running locally (where it is useful and doesn’t generate excessive output). - Pass optimization flags for scalac down to partest, using the new partest version 1.0.15\6. - Call the new sbt-based PR validation from `scripts/jobs/validate/test`. - Disable the tests `run/t7843-jsr223-service` and `run/t7933` from scala#4959 for now. We need to set up a new test project (either partest or junit) that can run them on a packaged version of Scala, or possibly move them into a separate project that would naturally run from a packaged Scala as part of the community build.

szeiger · 2016-06-14T14:11:43Z

Offending jarlister tests disabled, updated to a new partest 1.0.16, MiMa comments added.

lrytz · 2016-06-19T11:40:05Z

LGTM! 🎈

szeiger added this to the 2.12.0-M5 milestone May 25, 2016

szeiger self-assigned this May 25, 2016

szeiger changed the title ~~Use sbt for PR validation~~ Use sbt for PR validation [ci: last-only] May 25, 2016

szeiger mentioned this pull request May 30, 2016

when running tests during PR validation, use sbt not ant [ci: last-only] #4933

Closed

szeiger mentioned this pull request Jun 3, 2016

finish the sbt build scala/scala-dev#45

Closed

20 tasks

adriaanm reviewed Jun 3, 2016
View reviewed changes

dwijnand mentioned this pull request Jun 4, 2016

Remove experimental status from sbt build in load message #5211

Merged

szeiger mentioned this pull request Jun 6, 2016

Parse -Dpartest.scalac_opts in SBT runner scala/scala-partest#64

Merged

lrytz reviewed Jun 7, 2016
View reviewed changes

szeiger force-pushed the wip/validate-test-use-sbt branch 3 times, most recently from 869e10a to 9167173 Compare June 10, 2016 16:33

szeiger force-pushed the wip/validate-test-use-sbt branch from 9167173 to 5aa79ad Compare June 14, 2016 10:39

dwijnand reviewed Jun 14, 2016
View reviewed changes

szeiger force-pushed the wip/validate-test-use-sbt branch from 5aa79ad to f2d0f1e Compare June 14, 2016 14:10

Remove bnd warnings from log output

c2c08a4

szeiger mentioned this pull request Jun 17, 2016

Temporarily insource Scalacheck #5229

Closed

scala-jenkins added the reviewed label Jun 19, 2016

lrytz merged commit aaf7bc0 into scala:2.11.x Jun 19, 2016

adriaanm added 2.12 and removed 2.12 labels Oct 29, 2016

dwijnand mentioned this pull request Apr 25, 2017

Fix MiMa feature request URL #5863

Merged

Use sbt for PR validation [ci: last-only] #5190

Use sbt for PR validation [ci: last-only] #5190

Uh oh!

Conversation

szeiger commented May 25, 2016

Uh oh!

szeiger commented May 25, 2016

Uh oh!

retronym commented May 26, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

retronym commented May 26, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

szeiger commented May 30, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented May 30, 2016

Uh oh!

szeiger commented May 30, 2016

Uh oh!

szeiger commented Jun 3, 2016

Uh oh!

szeiger commented Jun 3, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adriaanm commented Jun 4, 2016

Uh oh!

retronym commented Jun 4, 2016

Uh oh!

retronym commented Jun 4, 2016

Uh oh!

szeiger commented Jun 6, 2016

Uh oh!

adriaanm commented Jun 6, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szeiger commented Jun 8, 2016

Uh oh!

lrytz commented Jun 8, 2016

Uh oh!

retronym commented Jun 9, 2016

Uh oh!

szeiger commented Jun 9, 2016

Uh oh!

szeiger commented Jun 9, 2016

Uh oh!

adriaanm commented Jun 10, 2016

Uh oh!

szeiger commented Jun 10, 2016

Uh oh!

szeiger commented Jun 13, 2016

Uh oh!

adriaanm commented Jun 13, 2016

Uh oh!

szeiger commented Jun 14, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szeiger commented Jun 14, 2016

Uh oh!

lrytz commented Jun 19, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

retronym commented May 26, 2016 •

edited

Loading

retronym commented May 26, 2016 •

edited

Loading

szeiger commented May 30, 2016 •

edited

Loading