-
Notifications
You must be signed in to change notification settings - Fork 3.1k
FSC / REPL Bug Bonanza #4040
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FSC / REPL Bug Bonanza #4040
Conversation
The transformation of applications to specialized methods
relies on the owner of said method having had the specialization
info transform run which stashes a bunch of related data into
per-run caches such as `SpecializeTypes#{typeEnv}`.
Recently, we found that per-run caches didn't quite live up
to there name, and in fact weren't being cleaned up before
a new run. This was remedied in 00e11ff.
However, no good deed goes unpunished, and this led to a
regression in specialization in the REPL and FSC.
This commit makes two changes:
- change the specialization info tranformer to no longer
directly enter specialized methods into the `info` of whatever
the current phase happens to be. This stops them showing up
`enteringTyper` of the following run.
- change `adaptInfos` to simply discard all but the oldest
entry in the type history when bringing a symbol from one
run into the next. This generalizes the approach taken to
fix SI-7801. The specialization info transformer will now
execute in each run, and repopulate `typeEnv` and friends.
I see that we have a seemingly related bandaid for this sort
of problem since 08505bd. In a followup, I'll try to revert
that.
08505bd added a workaround for an undiagnosed interaction between resident compilation and specialzation. This is no longer needed after the previous commit.
We needed to hop from the enum's owner to its companion module in an early phase of the compiler. The enclosed test used to fail when this lookup returned NoSymbol on the second run of the resident compiler when triggered from `MixinTransformer`: the lookup didn't work after the flatten info transform. This is related to the fact that module classes are flattened into the enclosing package, but module accessors remain in the enclosing class.
|
Manually assigned to 2.11.4, we're too close to 2.11.3 for these changes. |
56e2ba9 to
a654060
Compare
|
PLS REBUILD/pr-scala@a654060 |
|
(kitty-note-to-self: ignore 58592471) |
|
PLS REBUILD/pr-scala@a654060 |
|
(kitty-note-to-self: ignore 58592990) |
|
I've dumped SI-6782 into the too hard basket for now. My WIP is noted in that ticket's JIRA comments. |
|
ping @lrytz |
|
I like the change, I think it's the right thing to do. It probably fixes more issues that went undiscovered until now. I'm just wondering about performance for the IDE. Does the IDE create a new run for each typecheck? That would be quite often, and after this change all info transformers would run again on all symbols that are involved. WDYT? |
|
The IDE only runs to the typer phase, so unless something during typer is travelling forward through the phases, there should be no difference. |
|
LGTM - of course :) |
Under resident compilation, we were getting multiple copies of static forwarders created for mixed in methods. It seems like the bug was fixed as a by-produce of scala#4040. This commit adds a test to show this. I've confirmed that the test fails appropriately with 2.11.4. For future reference, before I figured out how to write the test for this one (test/resident doesn't seem to let you run the code after compiling it), I was using bash to test as follows: (export V=2.11.x; (scalac-hash $V sandbox/t5938_1.scala; (for i in 1 2; do echo sandbox/t5938.scala; done; printf '\n') | scalac-hash $V -Xresident); stty echo; scala-hash $V X ; echo ':javap -public X' | scala-hash $V);
Under resident compilation, we were getting multiple copies of static forwarders created for mixed in methods. It seems like the bug was fixed as a by-produce of scala#4040. This commit adds a test to show this. I've confirmed that the test fails appropriately with 2.11.4. For future reference, before I figured out how to write the test for this one (test/resident doesn't seem to let you run the code after compiling it), I was using bash to test as follows: (export V=2.11.x; (scalac-hash $V sandbox/t5938_1.scala; (for i in 1 2; do echo sandbox/t5938.scala; done; printf '\n') | scalac-hash $V -Xresident); stty echo; scala-hash $V X ; echo ':javap -public X' | scala-hash $V);
Under resident compilation, we were getting multiple copies of static forwarders created for mixed in methods. It seems like the bug was fixed as a by-produce of scala#4040. This commit adds a test to show this. I've confirmed that the test fails appropriately with 2.11.4. For future reference, before I figured out how to write the test for this one (test/resident doesn't seem to let you run the code after compiling it), I was using bash to test as follows: (export V=2.11.x; (scalac-hash $V sandbox/t5938_1.scala; (for i in 1 2; do echo sandbox/t5938.scala; done; printf '\n') | scalac-hash $V -Xresident); stty echo; scala-hash $V X ; echo ':javap -public X' | scala-hash $V);
Under resident compilation, we were getting multiple copies of static forwarders created for mixed in methods. It seems like the bug was fixed as a by-product of scala#4040. This commit adds a test to show this. I've confirmed that the test fails appropriately with 2.11.4. For future reference, before I figured out how to write the test for this one (test/resident doesn't seem to let you run the code after compiling it), I was using bash to test as follows: (export V=2.11.x; (scalac-hash $V sandbox/t5938_1.scala; (for i in 1 2; do echo sandbox/t5938.scala; done; printf '\n') | scalac-hash $V -Xresident); stty echo; scala-hash $V X ; echo ':javap -public X' | scala-hash $V);
The enclosed test case stopped working in 2.11.5 on the back of scala#4040. The key change was that we ran all post-typer info transformers on each run of the compiler, rather than trying to reuse the results of the previous run. In that patch, I noticed one place [1] in specialization that aggressively entered specialized members into the owning scope, rather than relying on `transformInfo` to place the new members in the scope of the newly created element of the info history. I made that change after noticing that this code could actually mutated scopes of specializaed types at the parser phase, which led to fairly obscure failures. This bug is another one of these obscure failures, and has the same root cause. We effectively "double specialiaze" Function0, which trips an assertion when `method apply$mcI$sp` is found twice in a scope. I have found another spot that was directly manipulating the scope, and removed the offending code.
The enclosed test case stopped working in 2.11.5 on the back of scala#4040. The key change was that we ran all post-typer info transformers on each run of the compiler, rather than trying to reuse the results of the previous run. In that patch, I noticed one place [1] in specialization that aggressively entered specialized members into the owning scope, rather than relying on `transformInfo` to place the new members in the scope of the newly created element of the info history. I made that change after noticing that this code could actually mutated scopes of specializaed types at the parser phase, which led to fairly obscure failures. This bug is another one of these obscure failures, and has the same root cause. We effectively "double specialiaze" Function0, which trips an assertion when `method apply$mcI$sp` is found twice in a scope. I have found another spot that was directly manipulating the scope, and removed the offending code. [1] scala#4040 (comment)
The enclosed test case stopped working in 2.11.5 on the back of scala#4040. The key change was that we ran all post-typer info transformers on each run of the compiler, rather than trying to reuse the results of the previous run. In that patch, I noticed one place [1] in specialization that aggressively entered specialized members into the owning scope, rather than relying on `transformInfo` to place the new members in the scope of the newly created element of the info history. I made that change after noticing that this code could actually mutated scopes of specializaed types at the parser phase, which led to fairly obscure failures. This bug is another one of these obscure failures, and has the same root cause. We effectively "double specialiaze" Function0, which trips an assertion when `method apply$mcI$sp` is found twice in a scope. I have found another spot that was directly manipulating the scope, and removed the offending code. [1] scala#4040 (comment)
The enclosed test case stopped working in 2.11.5 on the back of scala#4040. The key change was that we ran all post-typer info transformers on each run of the compiler, rather than trying to reuse the results of the previous run. In that patch, I noticed one place [1] in specialization that aggressively entered specialized members into the owning scope, rather than relying on `transformInfo` to place the new members in the scope of the newly created element of the info history. I made that change after noticing that this code could actually mutated scopes of specializaed types at the parser phase, which led to fairly obscure failures. This bug is another one of these obscure failures, and has the same root cause. We effectively "double specialiaze" Function0, which trips an assertion when `method apply$mcI$sp` is found twice in a scope. I have found another spot that was directly manipulating the scope, and removed the offending code. [1] scala#4040 (comment)
The enclosed test case stopped working in 2.11.5 on the back of scala#4040. The key change was that we ran all post-typer info transformers on each run of the compiler, rather than trying to reuse the results of the previous run. In that patch, I noticed one place [1] in specialization that aggressively entered specialized members into the owning scope, rather than relying on `transformInfo` to place the new members in the scope of the newly created element of the info history. I made that change after noticing that this code could actually mutated scopes of specializaed types at the parser phase, which led to fairly obscure failures. This bug is another one of these obscure failures, and has the same root cause. We effectively "double specialiaze" Function0, which trips an assertion when `method apply$mcI$sp` is found twice in a scope. I have found another spot that was directly manipulating the scope, and removed the offending code. [1] scala#4040 (comment)
The big contribution here is to change
adaptInfostostart with a clean slate (a one element type history)
on subsequent runs. I did the same a while back for package
classes, this is an generalization of that idea. This ensures
that all info transformers are run, importantly the side-effecty
ones.
This fixes a regression in specialization in the REPL, which
was a latent bug uncovered by our recent fix to
perRunCaches.The other changes are similarly themed, being more disciplined
when reading or mutating the type history.
Submitting for a test run and initial review by @lrytz. The commit
messages might need some fleshing out as I've been a little bit
hand wavy in a few of my explanations where I wasn't able to
precisely explain why the fix works.