Skip to content

Commit f6f11a5

Browse files
authored
[Xamarin.Android.Build.Tasks] MAM Member Remapping? (#6591)
Fixes: dotnet/java-interop#867 Context: dotnet/java-interop@1f27ab5 Context: #6142 (comment) Context: #7020 Changes: dotnet/java-interop@843f3c7...1f27ab5 * dotnet/java-interop@1f27ab55: [Java.Interop] Type & Member Remapping Support (#936) * dotnet/java-interop@02aa54e0: [Java.Interop.Tools.JavaCallableWrappers] marshal method decl types (#987) * dotnet/java-interop@e7bacc37: [ci] Update azure-pipelines.yaml to Pin .NET 6.0.202 (#986) * dotnet/java-interop@fb94d598: [Java.Interop.Tools.JavaCallableWrappers] Collect overriden methods (#985) * dotnet/java-interop@3fcce746: [Java.Interop.{Dynamic,Export}] Nullable Reference Type support (#980) ~~ The Scenarios ~~ Our Java binding infrastructure involves looking up types and methods via [JNI][0], and assumes that types and methods won't "move" in an unexpected manner. Methods which move from a subclass to a superclass works transparently. Methods which are moved to an entirely different class causes us problems. Case in point: [desugaring][0], which can *move* Java types to places that our bindings don't expect. For example, [`Arrays.stream(T[])`][1] may be moved into a `DesugarArrays` class, or a default interface method `Example.m()` may be moved into an `Example$-CC` type. Java.Interop has not supported such changes, resulting in throwing a `Java.Lang.NoSuchMethodError` on Android versions where methods are not where we expect. Additionally, the [InTune Mobile Application Management][3] team needs a expanded type and member lookup mechanism in order to simplify how they maintain their product. Currently they make things work by rewriting IL, which can be brittle. ~~ Build actions ~~ To improve support for this, dotnet/java-interop#936 introduces new `virtual` methods into `Java.Interop.JniRuntime.JniTypeManager` which are called as part of type and member lookup, allowing `AndroidTypeManager` to participate in the type and member resolution process. `AndroidTypeManager` in turn needs to know what types and members can be remapped, and what they should be remapped to. Some of these can be algorithmic, such as pre-pending `Desugar` or appending `$-CC` for the Desugar case. The InTune use case involves a table, contained within the [Microsoft.Intune.MAM.Remapper.Tasks NuGet package][4]. Update `src/Xamarin.Android.Build.Tasks` to add a new `@(_AndroidRemapMembers)` Build action. This build action is not externally supported; it's to help test the feature. Files with this build action are XML files which control type and member remapping: <replacements> <replace-type from="android/app/Activity" to="com/microsoft/intune/mam/client/app/MAMActivity" /> <replace-method source-type="com/microsoft/intune/mam/client/app/MAMActivity" source-method-name="onCreate" source-method-signature="(Landroid/os/Bundle;)V" target-type="com/microsoft/intune/mam/client/app/MAMActivity" target-method-name="onMAMCreate" target-method-instance-to-static="false" /> </replacements> `//replacements/replace-method` is structured with each attribute corresponding to a member on the `JniRuntime.ReplacementMethodInfo` structure, in dotnet/java-interop@1f27ab55. * `//replace-method/@source-type` is `JniRuntime.ReplacementMethodInfo.SourceJniType` * `//replace-method/@source-method-name` is `JniRuntime.ReplacementMethodInfo.SourceJniMethodName` * `//replace-method/@source-method-signature` is `JniRuntime.ReplacementMethodInfo.SourceJniMethodSignature` * `//replace-method/@target-type` is `JniRuntime.ReplacementMethodInfo.TargetJniType` * `//replace-method/@target-method-name` is `JniRuntime.ReplacementMethodInfo.TargetJniMethodName` * `//replace-method/@target-method-signature` is `JniRuntime.ReplacementMethodInfo.TargetJniMethodSignature` This attribute is optional. * `//replace-method/@target-method-parameter-count` is `JniRuntime.ReplacementMethodInfo.TargetJniMethodParameterCount`. This attribute is optional. * `//replace-method/@target-method-instance-to-static` is `JniRuntime.ReplacementMethodInfo.TargetJniMethodIsStatic` `@source-type`, `@source-method-name`, and `@source-method-signature` combined serve as a "key" for looking up the associated `@target-*` information. Update `src/Xamarin.Android.Build.Tasks` to add a new `@(_AndroidMamMappingFile)` Build action. This build action is not externally supported; it's to help test the feature. Files with this build action are expected to be JSON documents which follow the current conventions of `remapping-config.json`, within the `Microsoft.Intune.MAM.Remapper.Tasks` NuGet package. This build action is not externally supported; this is currently for testing purposes. `@(_AndroidMamMappingFile)` files are processed at build time into `@(_AndroidRemapMembers)` XML files. During App builds, all `@(_AndroidRemapMembers)` files are merged into an `@(AndrodAsset)` named `xa-internal/xa-mam-mapping.xml`. This asset is opened and provided to `JNIEnv.Initialize()` as part of native app startup. ~~ Putting it all together ~~ This will only work on .NET 7+. App project has a `@(_AndroidRemapMembers)` file. This item is processed during App build, stored into the `.apk`, and read during app startup on an Android device. Given a Java binding such as: public partial class Activity { static readonly JniPeerMembers _members = new XAPeerMembers ("android/app/Activity", typeof (Activity)); } when the `JniPeerMembers` constructor runs, it will call `JniEnvironment.Runtime.TypeManager.GetReplacementType("android/app/Activity")`. If `@(_AndroidRemapMembers)` is based on the InTune `remapping-config.json` file, then `android/app/Activity` is mapped to `com/microsoft/intune/mam/client/app/MAMActivity`, and `JNIEnv::FindClass()` will be told to lookup `MAMActivity`, *not* `Activity`. *If `MAMActivity` can't be found*, e.g. you're testing this all out, the app will ~immediately crash, as `MAMActivity` doesn't exist. 😅 If `MAMActivity` can be found, eventually `Activity.OnCreate()` will need to be invoked: partial class Activity { protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState) { const string __id = "onCreate.(Landroid/os/Bundle;)V"; try { JniArgumentValue* __args = stackalloc JniArgumentValue [1]; __args [0] = new JniArgumentValue ((savedInstanceState == null) ? IntPtr.Zero : ((global::Java.Lang.Object) savedInstanceState).Handle); _members.InstanceMethods.InvokeVirtualVoidMethod (__id, this, __args); } finally { global::System.GC.KeepAlive (savedInstanceState); } } } `_members.InstanceMethods.InvokeVirtualVoidMethod()` will internally make a call similar to: var r = JniEnvironment.Runtime.TypeManager.GetReplacementMethodInfo ( "com/microsoft/intune/mam/client/app/MAMActivity", "onCreate", "(Landroid/os/Bundle;)V" ); The data returned will be equivalent to: var r = new JniRuntime.ReplacementMethodInfo { SourceJniType = "com/microsoft/intune/mam/client/app/MAMActivity", // from input parameter SourceJniMethodName = "onCreate", // from input parameter SourceJniMethodSignature = "(Landroid/os/Bundle;)V", // from input parameter TargetJniType = "com/microsoft/intune/mam/client/app/MAMActivity", // from //replace-method/@target-type TargetJniMethodName = "onMAMCreate", // from //replace-method/@target-method-name TargetJniMethodSignature = "(Landroid/os/Bundle;)V", // from input parameter, as signature didn't change TargetJniMethodParameterCount = 1, // computed based on signature TargetJniMethodIsStatic = false, // from //replace-method/@target-method-instance-to-static } This will allow `_members.InstanceMethods.InvokeVirtualVoidMethod()` to instead resolve and invoke `MAMActivity.onMAMCreate()`. ~~ Tools ~~ `tools/remap-mam-json-to-xml` is added, and will process the InTune JSON file into `@(_AndroidRemapMembers)` XML: $ dotnet run --project tools/remap-mam-json-to-xml -- \ $HOME/.nuget/packages/microsoft.intune.mam.remapper.tasks/0.1.4635.1/content/MonoAndroid10/remapping-config.json <replacements>… ~~ Unit Tests ~~ `@(_AndroidRemapMembers)` usage is required by `Mono.Android.NET-Tests.apk`, as `Java.Interop-Tests.dll` exercises the type and member remapping logic. ~~ Unrelated `gref+` logging fixes ~~ When `debug.mono.log` = `gref+`, the app could crash: signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x79c045dead This was likely because a constant string was provided to `OSBridge::_write_stack_trace()`, which tried to write into the constant string, promptly blowing things up. Workaround: don't use `gref+` logging when a GC occurs? (Fortunately, `gref+` logging isn't the default.) Better workaround: Don't Do That™. Don't write to const strings. ~~ About `@(_AndroidRemapMembers)` Semantics… ~~ 1. Changing the Java hierarchy "requires" changing the managed hierarchy to mirror it. If we rename `Activity` to `RemapActivity` but *don't* change `MainActivity` to inherit the (bound!) `Example.RemapActivity`, the app *crashes*: JNI DETECTED ERROR IN APPLICATION: can't call void example.RemapActivity.onMyCreate(android.os.Bundle) on instance of example.MainActivity This can be "fixed" *without* changing the base class of `MainActivity` by instead changing the base class of the Java Callable Wrapper for `MainActivity` to `example.RemapActivity`. This can be done manually (just edit the files in `obj/…`!), but isn't really supported in "normal" xamarin-android usage (the next Clean will wipe your changes). Presumably InTune would make this Just Work by e.g. patching the `MainActivity.class` file. 2. `/replacements/replace-type` interacts with `/replacements/replace-method`: at runtime, `//replace-type@from` *no longer exists*, meaning you ***cannot*** use that name in `//replace-method/@source-type` either! If `Activity` is remapped to `RemapActivity`, then *there is no* `Activity.onCreate()` method to similarly remap. Instead, you need to specify `RemapActivity.onCreate()`. This warps the brain a bit. This: <replace-method source-type="example/RemapActivity" source-method-name="onCreate" target-type="example/RemapActivity" target-method-name="onMyCreate" target-method-instance-to-static="false" /> not this: <replace-method source-type="android/app/Activity" source-method-name="onCreate" target-type="example/RemapActivity" target-method-name="onMyCreate" target-method-instance-to-static="false" /> 3. Don't intermix type renames with `/replace-method/@target-method-instance-to-static='true']`. It *can* be done, but also warps the brain. The deal with `@target-method-instance-to-static` is that it it changes the target method signature -- unless explicitly provided in `/replace-method/@target-method-signature` -- so that the "source declaring type" is a prefix. Thus given <replace-method source-type="android/view/View" source-method-name="setOnClickListener" target-type="example/ViewHelper" target-method-name="mySetOnClickListener" target-method-instance-to-static="true" /> we'll look for `ViewHelper.mySetOnClickListener(View, View.OnClickListener)`. If we renamed `View` to `MyView`, we would instead look for `ViewHelper.mySetOnClickListener(MyView, View.OnClickListener)` (note changed parameter type). This almost certainly *won't* work. ~~ InTune Integration Testing? ~~ For "more complete" InTune integration testing, one will want the path to `remapping-config.json`, without hardcoding things. This can be done with `%(PackageReference.GeneratePathProperty)`=True and using `$(PkgMicrosoft_Intune_MAM_Remapper_Tasks)`: <ItemGroup> <PackageReference Include="Microsoft.Intune.MAM.Remapper.Tasks" Version="0.1.4635.1" IncludeAssets="none" GeneratePathProperty="True" ReferenceOutputAssembly="False" /> </ItemGroup> <Target Name="_AddMamFiles" BeforeTargets="_AddAndroidCustomMetaData"> <ItemGroup> <_AndroidMamMappingFile Include="$(PkgMicrosoft_Intune_MAM_Remapper_Tasks)/content/MonoAndroid10/remapping-config.json" /> </ItemGroup> </Target> This is still fraught with some peril, as it likely also depends on getting the right "inner" build, which may require using the plural `$(TargetFrameworks)` property, not the singular `$(TargetFramework)`. This might still be a useful start. ~~ TODO ~~ Optimize this mess: #7020 [0]: https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html [1]: https://developer.android.com/studio/write/java8-support#library-desugaring [2]: https://developer.android.com/reference/java/util/Arrays#stream(T[]) [3]: https://docs.microsoft.com/en-us/mem/intune/fundamentals/what-is-intune [4]: https://www.nuget.org/packages/Microsoft.Intune.MAM.Remapper.Tasks/
1 parent 9fd37e3 commit f6f11a5

File tree

35 files changed

+1160
-62
lines changed

35 files changed

+1160
-62
lines changed

Documentation/workflow/DevelopmentTips.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,18 @@
22

33
Tips and tricks while developing Xamarin.Android.
44

5+
# Run MSBuild-Based On-Device Unit Tests
6+
7+
The [`tests/MSBuildDeviceIntegration`](tests/MSBuildDeviceIntegration)
8+
directory contains NUnit-based unit tests which need to run against an attached
9+
Android device (hardware or emulator). There are *lots* of tests in here, and
10+
running them all can take a significant amount of time.
11+
12+
If you need to run only *one* `[Test]` method, you can use
13+
[`dotnet test --filter`](https://docs.microsoft.com/dotnet/core/testing/selective-unit-tests?pivots=mstest):
14+
15+
./dotnet-local.sh test bin/TestDebug/MSBuildDeviceIntegration/net6.0/MSBuildDeviceIntegration.dll --filter "Name~TypeAndMemberRemapping"
16+
517
# Update directory
618

719
When a Xamarin.Android app launches on an Android device, and the app was

external/Java.Interop

src/Mono.Android/Android.Runtime/AndroidRuntime.cs

Lines changed: 158 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
using System.Linq;
55
using System.Runtime.CompilerServices;
66
using System.Runtime.InteropServices;
7+
using System.Runtime.Versioning;
78
using System.Text;
89
using System.Threading;
910
using System.Reflection;
@@ -263,7 +264,11 @@ protected override IEnumerable<Type> GetTypesForSimpleReference (string jniSimpl
263264
{
264265
string? j = JNIEnv.TypemapManagedToJava (type);
265266
if (j != null) {
266-
return j;
267+
return
268+
#if NET
269+
GetReplacementTypeCore (j) ??
270+
#endif // NET
271+
j;
267272
}
268273
if (JNIEnv.IsRunningOnDesktop) {
269274
return JavaNativeTypeManager.ToJniName (type);
@@ -274,14 +279,163 @@ protected override IEnumerable<Type> GetTypesForSimpleReference (string jniSimpl
274279
protected override IEnumerable<string> GetSimpleReferences (Type type)
275280
{
276281
string? j = JNIEnv.TypemapManagedToJava (type);
282+
#if NET
283+
j = GetReplacementTypeCore (j) ?? j;
284+
#endif // NET
285+
if (JNIEnv.IsRunningOnDesktop) {
286+
string? d = JavaNativeTypeManager.ToJniName (type);
287+
if (j != null && d != null) {
288+
return new[]{j, d};
289+
}
290+
if (d != null) {
291+
return new[]{d};
292+
}
293+
}
277294
if (j != null) {
278-
yield return j;
295+
return new[]{j};
279296
}
280-
if (JNIEnv.IsRunningOnDesktop) {
281-
yield return JavaNativeTypeManager.ToJniName (type);
297+
return Array.Empty<string> ();
298+
}
299+
300+
#if NET
301+
protected override IReadOnlyList<string>? GetStaticMethodFallbackTypesCore (string jniSimpleReference)
302+
{
303+
ReadOnlySpan<char> name = jniSimpleReference;
304+
int slash = name.LastIndexOf ('/');
305+
var desugarType = new StringBuilder (jniSimpleReference.Length + "Desugar".Length);
306+
if (slash > 0) {
307+
desugarType.Append (name.Slice (0, slash+1))
308+
.Append ("Desugar")
309+
.Append (name.Slice (slash+1));
310+
} else {
311+
desugarType.Append ("Desugar").Append (name);
312+
}
313+
314+
return new[]{
315+
desugarType.ToString (),
316+
$"{jniSimpleReference}$-CC"
317+
};
318+
}
319+
320+
protected override string? GetReplacementTypeCore (string jniSimpleReference)
321+
{
322+
if (JNIEnv.ReplacementTypes == null) {
323+
return null;
324+
}
325+
if (JNIEnv.ReplacementTypes.TryGetValue (jniSimpleReference, out var v)) {
326+
return v;
327+
}
328+
return null;
329+
}
330+
331+
protected override JniRuntime.ReplacementMethodInfo? GetReplacementMethodInfoCore (string jniSourceType, string jniMethodName, string jniMethodSignature)
332+
{
333+
if (JNIEnv.ReplacementMethods == null) {
334+
return null;
335+
}
336+
#if !STRUCTURED
337+
if (!JNIEnv.ReplacementMethods.TryGetValue (CreateReplacementMethodsKey (jniSourceType, jniMethodName, jniMethodSignature), out var r) &&
338+
!JNIEnv.ReplacementMethods.TryGetValue (CreateReplacementMethodsKey (jniSourceType, jniMethodName, GetMethodSignatureWithoutReturnType ()), out r) &&
339+
!JNIEnv.ReplacementMethods.TryGetValue (CreateReplacementMethodsKey (jniSourceType, jniMethodName, null), out r)) {
340+
return null;
341+
}
342+
ReadOnlySpan<char> replacementInfo = r;
343+
344+
var targetType = GetNextString (ref replacementInfo);
345+
var targetName = GetNextString (ref replacementInfo);
346+
var targetSig = GetNextString (ref replacementInfo);
347+
var paramCountStr = GetNextString (ref replacementInfo);
348+
var isStaticStr = GetNextString (ref replacementInfo);
349+
350+
int? paramCount = null;
351+
if (!paramCountStr.IsEmpty) {
352+
if (!int.TryParse (paramCountStr, 0, System.Globalization.CultureInfo.InvariantCulture, out var count)) {
353+
return null;
354+
}
355+
paramCount = count;
356+
}
357+
358+
bool isStatic = false;
359+
if (isStaticStr.Equals ("true", StringComparison.Ordinal)) {
360+
isStatic = true;
361+
}
362+
363+
if (targetSig.IsEmpty && isStatic) {
364+
paramCount = paramCount ?? JniMemberSignature.GetParameterCountFromMethodSignature (jniMethodSignature);
365+
paramCount++;
366+
jniMethodSignature = $"(L{jniSourceType};" + jniMethodSignature.Substring ("(".Length);
367+
}
368+
369+
return new JniRuntime.ReplacementMethodInfo {
370+
SourceJniType = jniSourceType,
371+
SourceJniMethodName = jniMethodName,
372+
SourceJniMethodSignature = jniMethodSignature,
373+
TargetJniType = targetType.IsEmpty ? jniSourceType : new string (targetType),
374+
TargetJniMethodName = targetName.IsEmpty ? jniMethodName : new string (targetName),
375+
TargetJniMethodSignature = targetSig.IsEmpty ? jniMethodSignature : new string (targetSig),
376+
TargetJniMethodParameterCount = paramCount,
377+
TargetJniMethodInstanceToStatic = isStatic,
378+
};
379+
#else
380+
if (!JNIEnv.ReplacementMethods.TryGetValue ((jniSourceType, jniMethodName, jniMethodSignature), out var r) &&
381+
!JNIEnv.ReplacementMethods.TryGetValue ((jniSourceType, jniMethodName, GetMethodSignatureWithoutReturnType ()), out r) &&
382+
!JNIEnv.ReplacementMethods.TryGetValue ((jniSourceType, jniMethodName, null), out r)) {
383+
return null;
384+
}
385+
var targetSig = r.TargetSignature;
386+
var paramCount = r.ParamCount;
387+
if (targetSig == null && r.TurnStatic) {
388+
targetSig = $"(L{jniSourceType};" + jniMethodSignature.Substring ("(".Length);
389+
paramCount = paramCount ?? JniMemberSignature.GetParameterCountFromMethodSignature (jniMethodSignature);
390+
paramCount++;
391+
}
392+
return new JniRuntime.ReplacementMethodInfo {
393+
SourceJniType = jniSourceType,
394+
SourceJniMethodName = jniMethodName,
395+
SourceJniMethodSignature = jniMethodSignature,
396+
TargetJniType = r.TargetType ?? jniSourceType,
397+
TargetJniMethodName = r.TargetName ?? jniMethodName,
398+
TargetJniMethodSignature = targetSig ?? jniMethodSignature,
399+
TargetJniMethodParameterCount = paramCount,
400+
TargetJniMethodInstanceToStatic = r.TurnStatic,
401+
};
402+
#endif // !STRUCTURED
403+
404+
string GetMethodSignatureWithoutReturnType ()
405+
{
406+
int i = jniMethodSignature.IndexOf (')');
407+
return jniMethodSignature.Substring (0, i+1);
408+
}
409+
410+
string GetValue (string? value)
411+
{
412+
return value == null ? "null" : $"\"{value}\"";
413+
}
414+
415+
ReadOnlySpan<char> GetNextString (ref ReadOnlySpan<char> info)
416+
{
417+
int index = info.IndexOf ('\t');
418+
var r = info;
419+
if (index >= 0) {
420+
r = info.Slice (0, index);
421+
info = info.Slice (index+1);
422+
return r;
423+
}
424+
info = default;
425+
return r;
282426
}
283427
}
284428

429+
static string CreateReplacementMethodsKey (string? sourceType, string? methodName, string? methodSignature) =>
430+
new StringBuilder ()
431+
.Append (sourceType)
432+
.Append ('\t')
433+
.Append (methodName)
434+
.Append ('\t')
435+
.Append (methodSignature)
436+
.ToString ();
437+
#endif // NET
438+
285439
delegate Delegate GetCallbackHandler ();
286440

287441
static MethodInfo? dynamic_callback_gen;

src/Mono.Android/Android.Runtime/JNIEnv.cs

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,11 @@
1515
using Java.Interop.Tools.TypeNameMappings;
1616
using System.Diagnostics.CodeAnalysis;
1717

18+
#if NET
19+
using ReplacementTypesDict = System.Collections.Generic.Dictionary<string, string>;
20+
using ReplacementMethodsDict = System.Collections.Generic.Dictionary<string, string>;
21+
#endif // NET
22+
1823
namespace Android.Runtime {
1924
#pragma warning disable 0649
2025
struct JnienvInitializeArgs {
@@ -35,6 +40,8 @@ struct JnienvInitializeArgs {
3540
public int packageNamingPolicy;
3641
public byte ioExceptionType;
3742
public int jniAddNativeMethodRegistrationAttributePresent;
43+
public IntPtr mappingXml;
44+
public int mappingXmlLen;
3845
}
3946
#pragma warning restore 0649
4047

@@ -61,6 +68,11 @@ public static partial class JNIEnv {
6168
static AndroidRuntime? androidRuntime;
6269
static BoundExceptionType BoundExceptionType;
6370

71+
#if NET
72+
internal static ReplacementTypesDict? ReplacementTypes;
73+
internal static ReplacementMethodsDict? ReplacementMethods;
74+
#endif // NET
75+
6476
[ThreadStatic]
6577
static byte[]? mvid_bytes;
6678

@@ -166,6 +178,13 @@ internal static unsafe void Initialize (JnienvInitializeArgs* args)
166178
gref_class = args->grefClass;
167179
mid_Class_forName = new JniMethodInfo (args->Class_forName, isStatic: true);
168180

181+
#if NET
182+
if (args->mappingXml != IntPtr.Zero) {
183+
var xml = Encoding.UTF8.GetString ((byte*) args->mappingXml, args->mappingXmlLen);
184+
(ReplacementTypes, ReplacementMethods) = MamXmlParser.ParseStrings (xml);
185+
}
186+
#endif // NET
187+
169188
if (args->localRefsAreIndirect == 1)
170189
IdentityHash = v => _monodroid_get_identity_hash_code (Handle, v);
171190
else

0 commit comments

Comments
 (0)