Skip to content

Stable names for lambda lifted method and fresh names#6300

Merged
lrytz merged 5 commits intoscala:2.13.xfrom
retronym:topic/stable-fresh
May 31, 2018
Merged

Stable names for lambda lifted method and fresh names#6300
lrytz merged 5 commits intoscala:2.13.xfrom
retronym:topic/stable-fresh

Conversation

@retronym
Copy link
Copy Markdown
Member

@retronym retronym commented Feb 2, 2018

Fresh names are created using a FreshNameCreator, which appends
an increasing number to the given prefix.

scala> val fresh = new scala.reflect.internal.util.FreshNameCreator()
fresh: scala.reflect.internal.util.FreshNameCreator = scala.reflect.internal.util.FreshNameCreator@42b84286

scala> List("foo$", "bar$", "foo$").map(fresh.newName(_))
res1: List[String] = List(foo$1, bar$1, foo$2)

Each compilation unit had its own fresh name creator, which is used
in the regular compiler. Macros and quasiquotes make use of a global
creator (at least, as of #3401).

Both of these are too broadly scoped to help achieve deterministic
fresh names: if sources are recompiled in a different order or separately
recompiled, the fresh name counters can be different. Methods in a given
compilation unit are not necessarily typechecked in a linear fashion;
they might be typechecked ahead of time to provide an inferred type
to a caller.

This commit:

  • Changes all known fresh name creations within the typer phase (in which
    out-of-order typechecking is a factor) to use a fineer grained fresh
    name creator. How fine grained? A fresh name generated as some position
    p shares the fresh name generator scoped at the closest method or
    class that encloses that the outermost enclosing tree at the same
    position. This definition is designed to give a shared fresh name
    creator for all fresh names generated in macro1(macro2()), even if
    the fresh names are requiested from with a Typer in the macro enclosed
    by a synthetic method.
  • Changes macro fresh names to use the same fresh naming scheme as the regular
    typechecker. An opt-out compiler option allows the old behaviour, but I'm
    interested to find real-world cases where the new scheme actually causes
    a problem

In addition, a small change is made to lambda lift to lift local methods in the
order that they are encountered during traversal, rather than sorting them
based on Symbol.isLess (which include Symbol.id, an order-of-typechecking
dependent value).

References scala/scala-dev#405

@retronym
Copy link
Copy Markdown
Member Author

/synch

@retronym
Copy link
Copy Markdown
Member Author

/rebuild

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Feb 28, 2018

This causes a binary incompatiblity with partest which is built (unwisely IMO) with the full Scala optimizer enabled.

[test] [error] Uncaught exception when running partest: java.lang.NoSuchMethodError: scala.reflect.internal.util.Collections.$anonfun$distinctBy$1(Lscala/Function1;Lscala/collection/mutable/ListBuffer;Lscala/collection/mutable/Set;Ljava/lang/Object;)Ljava/lang/Object;
[test] sbt.ForkMain$ForkError: java.lang.NoSuchMethodError: scala.reflect.internal.util.Collections.$anonfun$distinctBy$1(Lscala/Function1;Lscala/collection/mutable/ListBuffer;Lscala/collection/mutable/Set;Ljava/lang/Object;)Ljava/lang/Object;
[test] 	at scala.tools.partest.nest.AbstractRunner.run(AbstractRunner.scala:160)
[test] 	at scala.tools.partest.sbt.PartestTask.execute(Framework.scala:43)
[test] 	at sbt.ForkMain$Run$2.call(ForkMain.java:296)
[test] 	at sbt.ForkMain$Run$2.call(ForkMain.java:286)
[test] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[test] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[test] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[test] 	at java.lang.Thread.run(Thread.java:748)

I'll release a new version of partest that only enables local optimizations.

retronym added a commit to retronym/sbt-scala-modules that referenced this pull request Mar 1, 2018
Otherwise we can inline bytecode public, but unstable, artifacts
from the standard library, as was see in:

scala/scala#6300 (comment)
@retronym retronym force-pushed the topic/stable-fresh branch 2 times, most recently from 273b5fd to bee824a Compare March 1, 2018 06:38
@retronym
Copy link
Copy Markdown
Member Author

retronym commented Mar 1, 2018

Rebased on top of a new version of partest.

@retronym retronym requested a review from adriaanm March 1, 2018 06:39
@retronym retronym force-pushed the topic/stable-fresh branch from bee824a to ee06025 Compare March 12, 2018 06:35
@retronym
Copy link
Copy Markdown
Member Author

Rebased on the yet-to-be-merged #6391 to speed up the rebuilds this needs.

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Mar 12, 2018

The test failures suggest that the test suite under ./scalacheck is being compiled against, and inlining from, ./library, but being executed with STARR on the classpath.

java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(Lscala/runtime/ObjectRef;Lscala/Function2;Ljava/lang/Object;)Ljava/lang/Object;
	at TreeMapTest$.$anonfun$new$110(treemap.scala:151)
	at TreeMapTest$.$anonfun$new$110$adapted(treemap.scala:150)
	at scala.Function1.$anonfun$andThen$1(Function1.scala:52)
> show scalacheck/autoScalaLibrary
[info] false
> show scalacheck/test:autoScalaLibrary
[info] false
> show scalacheck/test:full-classpath
[info] * Attributed(/Users/jz/code/scala/target/scalacheck/test-classes)
[info] * Attributed(/Users/jz/code/scala/build/quick/classes/scalacheck)
[info] * Attributed(/Users/jz/code/scala/build/quick/classes/library)
[info] * Attributed(/Users/jz/code/scala/build/quick/classes/reflect)
[info] * Attributed(/Users/jz/code/scala/build/quick/classes/compiler)
[info] * Attributed(/Users/jz/code/scala/build/quick/classes/scaladoc)
[info] * Attributed(/Users/jz/.ivy2/cache/org.apache.ant/ant/jars/ant-1.9.4.jar)
[info] * Attributed(/Users/jz/.ivy2/cache/org.apache.ant/ant-launcher/jars/ant-launcher-1.9.4.jar)
[info] * Attributed(/Users/jz/.ivy2/cache/org.scala-lang.modules/scala-asm/bundles/scala-asm-6.0.0-scala-1.jar)
[info] * Attributed(/Users/jz/.ivy2/cache/org.scala-lang.modules/scala-xml_2.13.0-M3/bundles/scala-xml_2.13.0-M3-1.1.0.jar)
[info] * Attributed(/Users/jz/.ivy2/cache/jline/jline/jars/jline-2.14.5.jar)
[info] * Attributed(/Users/jz/.ivy2/cache/org.scalacheck/scalacheck_2.13.0-M1/jars/scalacheck_2.13.0-M1-1.13.5.jar)
[info] * Attributed(/Users/jz/.ivy2/cache/org.scala-sbt/test-interface/jars/test-interface-1.0.jar)
[success] Total time: 0 s, completed 13/03/2018 9:20:12 AM
> show scalacheck/test:console
[info] Starting scala interpreter...
[info]
Welcome to Scala 2.13.0-M3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_162).
Type in expressions for evaluation. Or try :help.

scala> classOf[scala.collection.TraversableOnce[_]].getProtectionDomain.getCodeSource
res0: java.security.CodeSource = (file:/Users/jz/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.13.0-M3.jar <no signer certificates>)
diff --git a/test/scalacheck/treemap.scala b/test/scalacheck/treemap.scala
index 6978ca3145..b6bcbf13a2 100644
--- a/test/scalacheck/treemap.scala
+++ b/test/scalacheck/treemap.scala
@@ -6,7 +6,13 @@ import Arbitrary._
 import util._
 import Buildable._

+import scala.reflect.internal.util.ScalaClassLoader.URLClassLoader
+
 object TreeMapTest extends Properties("TreeMap") {
+  sys.error(
+    classOf[scala.collection.TraversableOnce[_]].getProtectionDomain.getCodeSource.toString + "\n" + getClass.getClassLoader + "\n" + getClass.getClassLoader.asInstanceOf[java.net.URLClassLoader].getURLs.toList.mkString("\n"))
+
+
   def genTreeMap[A: Arbitrary: Ordering, B: Arbitrary]: Gen[TreeMap[A, B]] =
     for {
       keys <- listOf(arbitrary[A])
Caused by: java.lang.RuntimeException: (file:/Users/jz/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.13.0-M3.jar <no signer certificates>)
URLClassLoader with NativeCopyLoader with RawResources(
  urls = List(/Users/jz/code/scala/target/scalacheck/test-classes, /Users/jz/code/scala/build/quick/classes/scalacheck, /Users/jz/code/scala/build/quick/classes/library, /Users/jz/code/scala/build/quick/classes/reflect, /Users/jz/code/scala/build/quick/classes/compiler, /Users/jz/code/scala/build/quick/classes/scaladoc, /Users/jz/.ivy2/cache/org.apache.ant/ant/jars/ant-1.9.4.jar, /Users/jz/.ivy2/cache/org.apache.ant/ant-launcher/jars/ant-launcher-1.9.4.jar, /Users/jz/.ivy2/cache/org.scala-lang.modules/scala-asm/bundles/scala-asm-6.0.0-scala-1.jar, /Users/jz/.ivy2/cache/org.scala-lang.modules/scala-xml_2.13.0-M3/bundles/scala-xml_2.13.0-M3-1.1.0.jar, /Users/jz/.ivy2/cache/jline/jline/jars/jline-2.14.5.jar, /Users/jz/.ivy2/cache/org.scalacheck/scalacheck_2.13.0-M1/jars/scalacheck_2.13.0-M1-1.13.5.jar, /Users/jz/.ivy2/cache/org.scala-sbt/test-interface/jars/test-interface-1.0.jar),
  parent = DualLoader(a = java.net.URLClassLoader@371cbcd7, b = java.net.URLClassLoader@6eebc39e),
  resourceMap = Set(app.class.path, boot.class.path),
  nativeTemp = /var/folders/b7/xcc2k0ln6ldcv247ffpy2d1w0000gp/T/sbt_cb70a61b/sbt_f916ad03
)
file:/Users/jz/code/scala/target/scalacheck/test-classes/
file:/Users/jz/code/scala/build/quick/classes/scalacheck
file:/Users/jz/code/scala/build/quick/classes/library/
file:/Users/jz/code/scala/build/quick/classes/reflect/
file:/Users/jz/code/scala/build/quick/classes/compiler/
file:/Users/jz/code/scala/build/quick/classes/scaladoc/
file:/Users/jz/.ivy2/cache/org.apache.ant/ant/jars/ant-1.9.4.jar
file:/Users/jz/.ivy2/cache/org.apache.ant/ant-launcher/jars/ant-launcher-1.9.4.jar
file:/Users/jz/.ivy2/cache/org.scala-lang.modules/scala-asm/bundles/scala-asm-6.0.0-scala-1.jar
file:/Users/jz/.ivy2/cache/org.scala-lang.modules/scala-xml_2.13.0-M3/bundles/scala-xml_2.13.0-M3-1.1.0.jar
file:/Users/jz/.ivy2/cache/jline/jline/jars/jline-2.14.5.jar
file:/Users/jz/.ivy2/cache/org.scalacheck/scalacheck_2.13.0-M1/jars/scalacheck_2.13.0-M1-1.13.5.jar
file:/Users/jz/.ivy2/cache/org.scala-sbt/test-interface/jars/test-interface-1.0.jar
	at scala.sys.package$.error(package.scala:27)
	at TreeMapTest$.<init>(treemap.scala:13)

This feels like a mole that I've tried to whack before.

@retronym
Copy link
Copy Markdown
Member Author

Forking fixes the classpath issue, but no longer reports failures:

> scalacheck/testOnly TreeMapTest
[info] Compiling 1 Scala source to /Users/jz/code/scala/target/scalacheck/test-classes...
[info] + TreeMap.splitAt: OK, passed 100 tests.
[info] + TreeMap.init/last identity: OK, passed 100 tests.
[info] ! TreeMap.faily: Falsified after 0 passed tests.
[info] > ARG_0: 0
[info] > ARG_0_ORIGINAL: -1
[info] + TreeMap.span identity: OK, passed 100 tests.
[info] + TreeMap.remove all: OK, passed 100 tests.
[info] + TreeMap.contains all: OK, passed 100 tests.
[info] + TreeMap.takeWhile: OK, passed 100 tests.
[info] + TreeMap.until is exclusive: OK, passed 100 tests.
[info] + TreeMap.dropWhile: OK, passed 100 tests.
[info] + TreeMap.drop: OK, passed 100 tests.
[info] + TreeMap.take: OK, passed 100 tests.
[info] + TreeMap.worst-case tree height is iterable: OK, passed 100 tests.
[info] + TreeMap.foreach/iterator consistency: OK, passed 100 tests.
[info] + TreeMap.sorted: OK, passed 100 tests.
[info] + TreeMap.from is inclusive: OK, passed 100 tests.
[info] + TreeMap.to is inclusive: OK, passed 100 tests.
[info] + TreeMap.head: OK, passed 100 tests.
[info] + TreeMap.remove single: OK, passed 100 tests.
[info] + TreeMap.slice: OK, passed 100 tests.
[info] + TreeMap.size: OK, passed 100 tests.
[info] + TreeMap.last: OK, passed 100 tests.
[info] + TreeMap.head/tail identity: OK, passed 100 tests.
[info] + TreeMap.toSeq: OK, passed 100 tests.
[info] + TreeMap.take/drop identity: OK, passed 100 tests.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 9 s, completed 13/03/2018 9:33:58 AM

71f9b43

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Mar 13, 2018

Reported our desire to leave STARR off the non-forked test classpath to @eed3si9n who is currently refactoring that part of SBT.

I also identified the SBT/Scalacheck problem in forked mode, I'll try to fix that to unblock this PR.

@eed3si9n
Copy link
Copy Markdown
Member

I opened sbt/sbt#4009. Hopefully this captures what's going on.

@retronym retronym force-pushed the topic/stable-fresh branch from ee06025 to e55bbd7 Compare March 18, 2018 04:24
@retronym
Copy link
Copy Markdown
Member Author

retronym commented Mar 18, 2018

Cherry-picked #6440 (submitted to 2.12.x) into this PR to unblock.

Marking as WIP because of the commits from in-flight PRs here, the actual code in this PR is ready for review.

@retronym retronym added the WIP label Mar 18, 2018
@retronym retronym force-pushed the topic/stable-fresh branch from e55bbd7 to 38a1e21 Compare March 18, 2018 04:29
@retronym
Copy link
Copy Markdown
Member Author

The tripwire I set in 00d1a5a has triggered for some tests (mostly around toolbox compilation, I think). So there are a few more problematic uses of fresh names that need attention.

@retronym retronym force-pushed the topic/stable-fresh branch 2 times, most recently from 5f4178d to d8ff23e Compare March 22, 2018 00:33
@retronym retronym removed the WIP label Mar 22, 2018
@adriaanm adriaanm removed this from the 2.13.0-M4 milestone Apr 30, 2018
retronym added 3 commits May 29, 2018 17:28
Fresh names are created using a FreshNameCreator, which appends
an increasing number to the given prefix.

```
scala> val fresh = new scala.reflect.internal.util.FreshNameCreator()
fresh: scala.reflect.internal.util.FreshNameCreator = scala.reflect.internal.util.FreshNameCreator@42b84286

scala> List("foo$", "bar$", "foo$").map(fresh.newName(_))
res1: List[String] = List(foo$1, bar$1, foo$2)
```

Each compilation unit had its own fresh name creator, which is used
in the regular compiler. Macros and quasiquotes make use of a global
creator (at least, as of scala#3401).

Both of these are too broadly scoped to help achieve deterministic
fresh names: if sources are recompiled in a different order or separately
recompiled, the fresh name counters can be different. Methods in a given
compilation unit are not necessarily typechecked in a linear fashion;
they might be typechecked ahead of time to provide an inferred type
to a caller.

This commit:

  - Changes all known fresh name creations within the typer phase (in which
    out-of-order typechecking is a factor) to use a fineer grained fresh
    name creator. How fine grained? A fresh name generated as some position
    `p` shares the fresh name generator scoped at the closest method or
    class that encloses that the outermost enclosing tree at the same
    position. This definition is designed to give a shared fresh name
    creator for all fresh names generated in `macro1(macro2())`, even if
    the fresh names are requiested from with a Typer in the macro enclosed
    by a synthetic method.
  - Changes macro fresh names to use the same fresh naming scheme as the regular
    typechecker. An opt-out compiler option allows the old behaviour, but I'm
    interested to find real-world cases where the new scheme actually causes
    a problem

In addition, a small change is made to lambda lift to lift local methods in the
order that they are encountered during traversal, rather than sorting them
based on `Symbol.isLess` (which include `Symbol.id`, an order-of-typechecking
dependent value).
I've used this to flush out the corner cases fixed in the previous commit.
@retronym retronym force-pushed the topic/stable-fresh branch from 7ab0f1e to c5cc71f Compare May 29, 2018 07:31
@szeiger
Copy link
Copy Markdown
Contributor

szeiger commented May 29, 2018

Yes, I already did the changes for Factory. But it's easy to accidentally (or simply to avoid the overhead) serialize closures. We still rely on this for views and for lazy lists.

@retronym retronym force-pushed the topic/stable-fresh branch from 6d0f950 to 6e40f4f Compare May 29, 2018 23:04
 Conflicts:
	test/junit/scala/SerializationStabilityTest.scala
@retronym retronym force-pushed the topic/stable-fresh branch from 6e40f4f to 20858b3 Compare May 31, 2018 04:08
@retronym
Copy link
Copy Markdown
Member Author

The build failure is just the macro annotation binary incompat on Travis for the second-to-last commit. It can be safely ignored.

@lrytz lrytz merged commit 0efb710 into scala:2.13.x May 31, 2018
Copy link
Copy Markdown
Contributor

@adriaanm adriaanm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Hmm, looks like I got interrupted before submitting my review :-/)

def macroExpand(typer: Typer, expandee: Tree, mode: Mode, pt: Type): Tree = {
// By default, use the current typer's fresh name creator in macros. The compiler option
// allows people to opt in to the old behaviour of Scala 2.12, which used a global fresh creator.
if (!settings.YmacroFresh.value) currentFreshNameCreator = typer.fresh
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we use -Xsource:2.12 instead?

def nextEnclosing(p: Context => Boolean): Context =
if (p(this)) this else outer.nextEnclosing(p)

final def outermostContextAtCurrentPos: Context = {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs a comment, can be taken from PR description (what does it compute, and why). Avoid having this drift, co-opted for other incompatible purposes.

@@ -0,0 +1,228 @@
package scala.tools.nsc
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 awesome way to test stability, but it could really use a comment

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Jun 7, 2018 via email

@adriaanm
Copy link
Copy Markdown
Contributor

adriaanm commented Jun 7, 2018

Thanks!

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Jun 7, 2018 via email

@retronym
Copy link
Copy Markdown
Member Author

retronym commented Jun 7, 2018 via email

@raboof
Copy link
Copy Markdown
Contributor

raboof commented Apr 14, 2019

This looks great!

We compiled Akka 2.5.22 with Scala 2.13.0-M5 (which contains this improvement). Here I see a difference between our published artifact and my (independent) local build that suggests there might be more sources of nondeterminism in this area.

The diffoscope report can be found at https://arnout.engelen.eu/rb/akka/2.13.0-M5/2.5.22/reproducible-builds-diffoscope-output-akka-actor_2.13.0-M5-2.5.22.jar/index.html - for example see akka/io/InetAddressDnsResolver.class

@raboof
Copy link
Copy Markdown
Contributor

raboof commented Apr 17, 2019

I see a difference between our published artifact and my (independent) local build

I tracked the change down and it was due to a compiler plugin, scalac seems fine! Sorry for the noise and great work here 😄

@adriaanm
Copy link
Copy Markdown
Contributor

Great! Thanks for investigating.

smarter added a commit to dotty-staging/dotty that referenced this pull request Jul 24, 2023
Context bounds are desugared into term parameters `evidence$N` and before this
commit, the `N` was chosen to be unique in the current compilation unit. This
isn't great because it means that adding a new definition with a context bound
in the middle of a file would change the desugaring of subsequent definitions
in the same file.

Even worse, when using incremental compilation we could end up with the same
context bound desugared with a different value of `N` on different compilation
runs because the order in which a compilation unit is traversed during Typer is
not fixed but depends on the how the units that are jointly compiled depend on
each other (as demonstrated by the `stable-ctx-bounds` test). This issue
affects all fresh names generated during Typer, but it is especially
problematic for context bounds because they're part of the API and renaming
a method parameter forces the recompilation of all files calling that method.

To fix this, we now only require context bounds parameters to have unique names
among all the parameters of the method. This matches how we already desugar
`def foo(using A, B)` into `def foo(x$1: A, x$2: B)` regardless of the context.

Note that fresh names used in other situations are still problematic for
deterministic compilation, most of the time they're not part of the API checked
by Zinc, but they they can still lead to overcompilation if they appear in an
inline def since the entire body of the inline def constitutes its API. In the
future, we should follow Scala 2 lead and only require names to be fresh at the
method level: scala/scala#6300 (The Scala 2 logic is
slightly more complex to handle macros, but I don't think that applies to Scala
3 macros), see scala#7661.

Fixes scala#18080.
smarter added a commit to dotty-staging/dotty that referenced this pull request Jul 24, 2023
Context bounds are desugared into term parameters `evidence$N` and before this
commit, the `N` was chosen to be unique in the current compilation unit. This
isn't great because it means that adding a new definition with a context bound
in the middle of a file would change the desugaring of subsequent definitions
in the same file.

Even worse, when using incremental compilation we could end up with the same
context bound desugared with a different value of `N` on different compilation
runs because the order in which a compilation unit is traversed during Typer is
not fixed but depends on the how the units that are jointly compiled depend on
each other (as demonstrated by the `stable-ctx-bounds` test). This issue
affects all fresh names generated during Typer, but it is especially
problematic for context bounds because they're part of the API and renaming
a method parameter forces the recompilation of all files calling that method.

To fix this, we now only require context bounds parameters to have unique names
among all the parameters of the method. This matches how we already desugar
`def foo(using A, B)` into `def foo(using x$1: A, x$2: B)` regardless of the
context.

Note that fresh names used in other situations are still problematic for
deterministic compilation. Most of the time they're not part of the API checked
by Zinc, but they can still lead to overcompilation if they appear in an
`inline def` since the entire body of the `inline def` constitutes its API. In
the future, we should follow Scala 2's lead and only require names to be fresh
at the method level: scala/scala#6300 (The Scala 2
logic is slightly more complex to handle macros, but I don't think that applies
to Scala 3 macros), see scala#7661.

Fixes scala#18080.
smarter added a commit to dotty-staging/dotty that referenced this pull request Jul 24, 2023
Context bounds are desugared into term parameters `evidence$N` and before this
commit, the `N` was chosen to be unique in the current compilation unit. This
isn't great because it means that adding a new definition with a context bound
in the middle of a file would change the desugaring of subsequent definitions
in the same file.

Even worse, when using incremental compilation we could end up with the same
context bound desugared with a different value of `N` on different compilation
runs because the order in which a compilation unit is traversed during Typer is
not fixed but depends on the how the units that are jointly compiled depend on
each other (as demonstrated by the `stable-ctx-bounds` test). This issue
affects all fresh names generated during Typer, but it is especially
problematic for context bounds because they're part of the API and renaming
a method parameter forces the recompilation of all files calling that method.

To fix this, we now only require context bounds parameters to have unique names
among all the parameters of the method. This matches how we already desugar
`def foo(using A, B)` into `def foo(using x$1: A, x$2: B)` regardless of the
context.

Note that fresh names used in other situations are still problematic for
deterministic compilation. Most of the time they're not part of the API checked
by Zinc, but they can still lead to overcompilation if they appear in an
`inline def` since the entire body of the `inline def` constitutes its API. In
the future, we should follow Scala 2's lead and only require names to be fresh
at the method level: scala/scala#6300 (The Scala 2
logic is slightly more complex to handle macros, but I don't think that applies
to Scala 3 macros), see scala#7661.

Fixes scala#18080.
smarter added a commit to dotty-staging/dotty that referenced this pull request Jul 25, 2023
Context bounds are desugared into term parameters `evidence$N` and before this
commit, the `N` was chosen to be unique in the current compilation unit. This
isn't great because it means that adding a new definition with a context bound
in the middle of a file would change the desugaring of subsequent definitions
in the same file.

Even worse, when using incremental compilation we could end up with the same
context bound desugared with a different value of `N` on different compilation
runs because the order in which a compilation unit is traversed during Typer is
not fixed but depends on the how the units that are jointly compiled depend on
each other (as demonstrated by the `stable-ctx-bounds` test). This issue
affects all fresh names generated during Typer, but it is especially
problematic for context bounds because they're part of the API and renaming
a method parameter forces the recompilation of all files calling that method.

To fix this, we now only require context bounds parameters to have unique names
among all the parameters of the method. This matches how we already desugar
`def foo(using A, B)` into `def foo(using x$1: A, x$2: B)` regardless of the
context.

Note that fresh names used in other situations are still problematic for
deterministic compilation. Most of the time they're not part of the API checked
by Zinc, but they can still lead to overcompilation if they appear in an
`inline def` since the entire body of the `inline def` constitutes its API. In
the future, we should follow Scala 2's lead and only require names to be fresh
at the method level: scala/scala#6300 (The Scala 2
logic is slightly more complex to handle macros, but I don't think that applies
to Scala 3 macros), see scala#7661.

Fixes scala#18080.
smarter added a commit to scala/scala3 that referenced this pull request Jul 25, 2023
Context bounds are desugared into term parameters `evidence$N` and
before this
commit, the `N` was chosen to be unique in the current compilation unit.
This
isn't great because it means that adding a new definition with a context
bound
in the middle of a file would change the desugaring of subsequent
definitions
in the same file.

Even worse, when using incremental compilation we could end up with the
same
context bound desugared with a different value of `N` on different
compilation
runs because the order in which a compilation unit is traversed during
Typer is
not fixed but depends on the how the units that are jointly compiled
depend on
each other (as demonstrated by the `stable-ctx-bounds` test). This issue
affects all fresh names generated during Typer, but it is especially
problematic for context bounds because they're part of the API and
renaming
a method parameter forces the recompilation of all files calling that
method.

To fix this, we now only require context bounds parameters to have
unique names
among all the parameters of the method. This matches how we already
desugar
`def foo(using A, B)` into `def foo(using x$1: A, x$2: B)` regardless of
the
context.

Note that fresh names used in other situations are still problematic for
deterministic compilation. Most of the time they're not part of the API
checked
by Zinc, but they can still lead to overcompilation if they appear in an
`inline def` since the entire body of the `inline def` constitutes its
API. In
the future, we should follow Scala 2's lead and only require names to be
fresh
at the method level: scala/scala#6300 (The Scala
2
logic is slightly more complex to handle macros, but I don't think that
applies
to Scala 3 macros), see #7661.

Fixes #18080.
Kordyjan pushed a commit to scala/scala3 that referenced this pull request Dec 1, 2023
Context bounds are desugared into term parameters `evidence$N` and before this
commit, the `N` was chosen to be unique in the current compilation unit. This
isn't great because it means that adding a new definition with a context bound
in the middle of a file would change the desugaring of subsequent definitions
in the same file.

Even worse, when using incremental compilation we could end up with the same
context bound desugared with a different value of `N` on different compilation
runs because the order in which a compilation unit is traversed during Typer is
not fixed but depends on the how the units that are jointly compiled depend on
each other (as demonstrated by the `stable-ctx-bounds` test). This issue
affects all fresh names generated during Typer, but it is especially
problematic for context bounds because they're part of the API and renaming
a method parameter forces the recompilation of all files calling that method.

To fix this, we now only require context bounds parameters to have unique names
among all the parameters of the method. This matches how we already desugar
`def foo(using A, B)` into `def foo(using x$1: A, x$2: B)` regardless of the
context.

Note that fresh names used in other situations are still problematic for
deterministic compilation. Most of the time they're not part of the API checked
by Zinc, but they can still lead to overcompilation if they appear in an
`inline def` since the entire body of the `inline def` constitutes its API. In
the future, we should follow Scala 2's lead and only require names to be fresh
at the method level: scala/scala#6300 (The Scala 2
logic is slightly more complex to handle macros, but I don't think that applies
to Scala 3 macros), see #7661.

Fixes #18080.

[Cherry-picked f322b7b]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants