Skip to content
This repository was archived by the owner on Nov 1, 2020. It is now read-only.

Conversation

@trylek
Copy link
Member

@trylek trylek commented May 16, 2019

As I haven't found out how to reopen my original PR after reverting it, I have prepared a new one. In the first commit I have pushed the original change rebased to the preparatory change moving CanInline
to CompilationModuleGroup. In the second commit I added logic for clean separation of input assembly set from the version bubble assembly set.

Earlier in CPAOT development, we used to use CompilationModuleGroup.ContainsType / ContainsMethodBody for checking whether a given typesystem element belongs to either of these sets. To facilitate the separation I have added two new methods on CompilationModuleGroup, VersionsWithType and VersionsWithMethodBody. I audited all calls to ContainsMethodBody & ContainsType in the CPAOT codebase and I modified those querying the version bubble to call VersionsWithType/MethodBody instead.

Thanks

Tomas

trylek added 2 commits May 14, 2019 17:25
As I don't see any way to revive the original PR I accidentally
merged in and then reverted, I have created a new one. In the initial
commit I have pushed the previous change. As the second commit
I have added the CompilationModuleGroup change suggested by
Simon - using separate sets for input assemblies vs. for the
version bubble.

This change modifies the existing --inputbubble CPAOT option
to work the same as Crossgen /largeversionbubble i.e. the main module
and all reference assemblies are considered to belong to the same
version bubble.

The gist of the change deals with encoding module overrides in
signatures. I have implemented a new R2R header table
ManifestMetadataTableNode and added logic for composing and emitting
the extra AssemblyRef list.

One challenging bit was that we need to emit all the signatures before
emitting the ManifestMetadataTableNode because only the emission
of the signatures shakes out all the required AssemblyRef's. I have
added a simple loop to ImportSectionsTableNode / ImportSectionNode
that traverses and emits all registered import cell signatures.

For instance entrypoints, I didn't find any manner of encoding
a module override for their signatures in the native hashtable so
for now I added filtering that we're only storing those instance
entrypoints that have the current input module as the reference
module. One can theoretically imagine that a module may contain
arbitrary generic instantiations spanning other modules when
larger bubbles are on.

While working on the change I also realized that the current way
of basing signature contexts on input modules in CorInfoImpl.ReadyToRun
was incorrect - when a method gets inlined, we still need the original
signature context because that's what defines the ambient module
used in the CoreCLR signature parser.

I have deleted the field and instead I added a new method
GetSignatureContext which currently returns
ReadyToRunCoregenNodeFactory.InputModuleContext. Once we start
implementing "single-file" (i.e. compiling multiple MSIL modules
into a single PE), we'll change this to something more fine-grained,
most likely based on some lookup on MethodBeingCompiled.

I found out that part of the framework libraries were crashing
in release build with large version bubble on due to the presence
of duplicate symbols. I found out that CPAOT was still inconsistent
w.r.t. means for formatting typesystem entities in AppendMangledName
methods. I went over all these methods and unified them to use
the name mangled for formatting types, methods and fields. As part
of this cleanup I could subsequently remove the WriteTo methods on
RuntimeDeterminedTypeHelper that are no longer needed.

Rerunning the Pri#1 tests with my new support for issues.targets
filtering out some annoying timing out GC tests I found out that
several tests failed during CPAOT compilation in the manifest
metadata table node. It turns out that the AssemblyRef metadata
table is not always unique, in particular I hit duplication of
the "mscorlib" reference in the table. This change makes CPAOT
resilient towards this type of issues.

Thanks

Tomas
Based on PR discussion I have modified the R2R single assembly
compilation module group to keep input assemblies separate from
the version bubble module set. With this change, CPAOT dependency
analysis no longer attempts to compose the code graph across all
assemblies, not only those being actually compiled, while keeping
the more relaxed inlining conditions in case of larger version
bubble. Based on this additional delta I could finally remove the
artificial filtering of methods in MethodEntryPointTableNode.

To facilitate this clean separation, I have added two new methods
to CompilationModuleGroup for the version bubble checks -
VersionsWithType and VersionsWithMethodBody. I have subsequently
audited all CompilationModuleGroup queries in the CPAOT compiler
and I changed the checks semantically referring to the version
bubble to use the new methods.

Thanks

Tomas
Copy link
Contributor

@nattress nattress left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I like the new module group API for version bubbling

versionBubbleModules.Add(module);

}
catch (Exception ex)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reasoning behind making a failure to resolve version bubble assemblies non-fatal? Intuitively, I would think if a path is mis-specified we'd want to die early.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, the problem is in the pre-existing script SimpleTest.targets that has no simple way of filtering out non-managed assemblies from the reference assembly set. In 2nd commit I have added a descriptive comment explaining this issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we catch the exception specific to that? I think we would still want to not silently move on for access denied, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, thanks, fixed in 4th commit.

trylek added 2 commits May 18, 2019 10:21
When determining reference module for encoding type
signatures, we're special-casing primitive types as
they don't need any reference module and use special
constants in the signature encoding scheme. I previously
didn't realize that object and string need the same
treatment. I have also added an explanatory comment to
Program.cs as to why we're using a try block when
opening reference modules.

Thanks

Tomas
@trylek trylek merged commit 7795aa0 into dotnet:master May 20, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants