Skip to content

Speed up partest#6413

Merged
retronym merged 9 commits intoscala:2.12.xfrom
retronym:topic/partest-apace-2.12.x
Mar 21, 2018
Merged

Speed up partest#6413
retronym merged 9 commits intoscala:2.12.xfrom
retronym:topic/partest-apace-2.12.x

Conversation

@retronym
Copy link
Copy Markdown
Member

(2.12.x version of #6391)

Prepares for, and then switches to, partest 1.1.6 which will
include a mode to run tests without forking.

Removes some resource hungry tests. Volunteers sought to revive
these in more frugal form.

Also backports the new test listener that integrates partest
into the Jenkins Test Results report.

 - Explicitly add test output path to the compiler classpath
   Rather than assuming its in the application classpath.
 - Ensure tests cleanup threads
 - Fork tests that inherently require it or ones that I can't
   figure out how to make work without it
Tests that take 30+ seconds to compile and/or execute
really add up to a productivity loss for contributors.

These tests served a purpose to show that the corresponding
changes were correct. But if we want them to serve an ongoing
purpose of guarding against regressions, they need to be
maintained to do so more quicky or be moved into a test suite
that is run less frequently.
@scala-jenkins scala-jenkins added this to the 2.12.6 milestone Mar 12, 2018
@retronym retronym mentioned this pull request Mar 12, 2018
@retronym retronym force-pushed the topic/partest-apace-2.12.x branch from 61ec1fb to a3e9512 Compare March 12, 2018 13:44
…ction

The logic that decides to print `Function`, rather than `scala.Function`
did not account for the multiplicity of symbols for a given package
in the JavaMirrors universe.
  - Update to partest that emits more detailed TestEvents
  - Group partest JUnit XML reports in, e.g, test.files.pos.xml
  - workaround Jenkins dislike of the work "run"
Requires a new version of partest to provide some missing metadata.

Sample files generated:

```
> ;partest --srcpath scaladoc --grep t7876; partest --grep default
...
```

```
⚡ (cd target/test/test-reports/partest && find . )
.
./test.files.jvm.xml
./test.files.neg.xml
./test.files.pos.xml
./test.files.presentation.xml
./test.files.run_.xml
./test.files.scalap.xml
./test.files.specialized.xml
./test.scaladoc.run_.xml
```
@retronym retronym force-pushed the topic/partest-apace-2.12.x branch from a3e9512 to 8c5930a Compare March 12, 2018 23:07
@retronym
Copy link
Copy Markdown
Member Author

My vote is to merge this and merge forward to 2.13. If we find a problem with Partest 1.1.7, we can just bump the version back without the other changes causing problems. @lrytz WDYT?

@retronym retronym requested a review from lrytz March 21, 2018 10:01
Copy link
Copy Markdown
Member

@lrytz lrytz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree to merge it here.

  • Why did you backport "Elide prefixes in printed types uniformly in runtime reflection"?
  • We could create a test/longrunning folder for the resource hungry tests, and run them in nightlies for example. WDYT?
  • Could you expand a little in the first commit message "Fork tests that inherently require it"? What are known conditions? For example, I randomly picked t8266-octal-interp.scala and was wondering why. Is it the mix of compiler an run output in the check file?

// `Settings` is used to check YdisableFlatCpCaching in ZipArchiveFlatClassPath
val factory = new ClassPathFactory(new Settings())
val containers = factory.classesInExpandedPath(Defaults.javaUserClassPath)
val containers = factory.classesInExpandedPath(sys.props("partest.output") + ":" + Defaults.javaUserClassPath)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably use java.io.File.pathSeparator to also work on windoze

if (getClass.getClassLoader.getParent != null) {
s.classpath.value = s.classpath.value match {
case "" => testOutput.toString
case s => s + ":" + testOutput.toString
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also java.io.File.pathSeparator

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pushed an extra commit

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pushed

don't see it yet 🙈

@retronym
Copy link
Copy Markdown
Member Author

Why did you backport "Elide prefixes in printed types uniformly in runtime reflection"?

Prior to that fix, tests that relied on Type.toString from runtime.universe would change depending on classloader details that differed between forked and unforked test execution.

We could create a test/longrunning folder for the resource hungry tests, and run them in nightlies for example. WDYT?

I don't think there is enough value in the ones I've dropped to setup that infrastructure.

Fork tests that inherently require it

Can do. Once I got down to the last few tests, I wasn't always sure why the forking was necessary, but for some I should be able to give an explanation.

@lrytz
Copy link
Copy Markdown
Member

lrytz commented Mar 21, 2018

Fork tests that inherently require it

Can do. Once I got down to the last few tests, I wasn't always sure why the forking was necessary, but for some I should be able to give an explanation.

Just a comment here is fine, instead of rebasing and rebuilding the whole thing. It doesn't need to be exhausitive, just a few pointers.

@retronym
Copy link
Copy Markdown
Member Author

Know reasons to fork:

  • testing @native (test/files/jvm/natives)
  • tests for our facade over JVM shutdown hooks

Less well known:

  • tests for memory leaks that are somewhat sensitive to the heap size and seemed more stable to partition into a separate VM
  • using less well known features of partest like including methvsfield.jar on the app classpath (**/methvsfield.*)

Not known/remembered:

  • test/files/run/t8266-octal-interp

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Mar 21, 2018

don't see it yet 🙈

🐵

@retronym retronym merged commit df4a883 into scala:2.12.x Mar 21, 2018
@retronym
Copy link
Copy Markdown
Member Author

Submitted the forward merge as #6451

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants