Skip to content

Commit d8d9e97

Browse files
authored
[pigeon] Reject unsupported native interop features and fix generated code that doesn't compile (#13153)
Pigeon accepted several combinations of `swift_use_ffi` or `kotlin_use_jni` with other features and options that generated code that doesn't compile. One combination crashed pigeon. flutter/flutter#193881 reported two of them: event channels and ProxyApis with Swift FFI. To find the rest, I generated every repo pigeon input and a set of edge-case inputs with each interop option. I then type-checked the Swift, compiled the Kotlin and analyzed the Dart. That found 19 broken combinations, and fixing them found a few more. This PR adds a validation error where native interop has no implementation yet, and fixes the generators for everything else. It also adds a test API to the test plugin that uses the names JNIgen renames, so the integration tests check pigeon's copy of JNIgen's renaming rules at runtime. **New validation errors.** Apart from event channels and ProxyApis with Kotlin JNI, none of these combinations ever produced code that compiled. - **Swift FFI:** - Event channels, ProxyApis and sealed classes. The error says to move them to a separate pigeon file that doesn't use `swift_use_ffi`. - Data class fields that conflict with `NSObject` members, because FFI bridge classes extend `NSObject`. Examples are `hash`, `isProxy` and `superclass`, plus names that ffigen renames, such as `alloc` or `isEqual`. - HostApi parameters named `wrappedError`. ffigen would generate two `wrappedError:` named parameters. - `includeErrorClass: false`. swiftgen compiles each Swift file on its own, so the file must declare its error class. Give each file a different `errorClassName` instead. - **Kotlin JNI:** Event channels, ProxyApis and sealed classes, the same as Swift FFI. The error says to move them to a separate pigeon file that doesn't use `kotlin_use_jni`. - Sealed classes never worked. The Kotlin compiles, but the generated Dart has no JNI conversion for them. - Event channels and ProxyApis did compile, because the Kotlin file falls back to channel code for them. They're rejected so that both interop options support the same features until these work over native interop. A file that generated before can now fail validation, but native interop is still marked experimental. - **Dart:** API method names that conflict with generated native interop members: - HostApi: `getInstance` and `createWithNativeInteropApi`. - FlutterApi: `implement`, and with JNI also `dartApi` and `register`. **Generator fixes** - **Swift (FFI):** - The bridge for a class without fields didn't compile because its `init()` was missing `override`. - `@asyncCallback` host methods: the bridge `await`ed a callback-style method. It now wraps the call in `withCheckedThrowingContinuation`. - `@SwiftFunction` overloads that differ only in nullability were ambiguous. The bridge now declares the result type. - An API method named `type` broke error handling in every method of that API, because `type(of:)` resolved to the method. The bridge now uses `Swift.type(of:)`. - Parameters that shared a name with one of the bridge's own parameters, such as a FlutterApi parameter named `error`, collided with it. Internal parameter names now end in `Arg`. Selectors don't change. - Two FFI files in the same module both declared `_PigeonFfiCodec`. It's now `private`. - **Dart:** - Pigeon crashed with `Bad state: No element` when a class without fields was used with Swift FFI. - JNI calls used the Kotlin member names, but JNIgen changes some of them: - It renames members that clash with its own, such as `type`, `release`, `use` and `isNull`. - It turns `getX()`, `isX()` and `setX()` into properties. - It turns `is`-prefixed data class properties that aren't `Boolean` into methods. A new `_JniBindingNamer` mirrors the JNIgen 1.0.1 renamer for data class fields, HostApi methods and FlutterApi overrides. It replaces `_getJniFieldName`, which handled only a few field names. - FFI calls assumed that Swift's Objective-C selectors always join the method name and the first argument label with `With`. Swift leaves out `With` when the method name ends with a preposition or the label starts with one. For example, `signIn(value:)` becomes `signInValue:` and `init(forKey:)` becomes `initForKey:`. Dart now applies the same rule to host API calls, FlutterApi callbacks and data class bridge initializers. It uses the preposition list from Swift's [`PartsOfSpeech.def`](https://github.com/swiftlang/swift/blob/main/lib/Basic/PartsOfSpeech.def) and Swift's camel-case word splitting. - Generated local variables shadowed parameters with the same name, such as `res`, `error`, `completer` and `response`. Generated locals now use the `pigeonVar_` prefix, and callback parameters use `arg_`. - Lists and maps of ProxyApis caused the interop codecs to contain the literal text `There is something wrong, a type is not classified`. The codecs now skip those types. - The class-name prefix for FFI types didn't match the Swift one in three cases: - The Swift file name wasn't UpperCamelCase. - `fileSpecificClassNameComponent` wasn't UpperCamelCase. - `SwiftOptions.fileSpecificClassNameComponent` was set. Dart now uses the same precedence and casing as the Swift generator. - Test files (`dartTestOut`) contained native interop code that referenced bindings they don't import. They're now generated with method channels only, which is all their mock handlers use. - **Docs:** The `includeErrorClass` docs on `SwiftOptions` and `KotlinOptions` explain the native interop requirements. **Test plugin** `native_interop_tests.dart` adds `NativeInteropReservedNames`, `NativeInteropReservedNamesHostApi` and `NativeInteropReservedNamesFlutterApi`. Their names cover each JNIgen renaming rule: - members that clash with JNIgen's own, such as `release`, `use`, `type`, `isNull`, `reference` and `jClass`; - getters, setters and `is` methods that JNIgen turns into properties, including `getDefault`, which becomes a Dart keyword; - `is` fields that aren't `bool`, which JNIgen turns into methods; - an `@async` method, which JNIgen keeps as a method because it's a Kotlin `suspend` function; - a FlutterApi method named `implementIn`, which clashes with JNIgen's `implementIn`. The Kotlin and Swift test plugins implement both APIs, and new integration tests call every method and round-trip the data class. The tests use `true` for `isNull` and `isReleased`, so they fail if the generated code calls `JObject`'s own members instead, which return `false`. This API is also how I found the Swift selector bug above: `implementIn` ends with a preposition. The test plugin and the native interop example are regenerated. Apart from the new test API, their diffs contain only: - the renames above; - `private` on `_PigeonFfiCodec`; - `Swift.type(of:)`; - the matching ffigen bindings and Objective-C header. Selectors and public APIs don't change. **Testing** - New unit tests: 12 for validation, 6 for the Dart generator and 4 for the Swift generator. All 581 pigeon unit tests pass. - The Android (emulator) and macOS integration suites pass, including the new reserved-name tests: 674 tests on Android, and 673 on macOS with 1 existing skip. I didn't run iOS locally. - I compiled Kotlin that uses every JNIgen-reserved name and ran JNIgen 1.0.1 on it. The generated Dart analyzes against those bindings with no errors. - I checked Swift's selector rule with `swiftc`, for every preposition and for names with underscores, digits and acronyms. - I re-ran the full probe. Validation rejects exactly the unsupported combinations, and the generated Swift for every other case type-checks with `swiftc`. Fixes flutter/flutter#193881 ## Pre-Review Checklist **Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed. [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
1 parent 1f55bfa commit d8d9e97

25 files changed

Lines changed: 19026 additions & 9842 deletions

‎packages/pigeon/CHANGELOG.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,27 @@
1+
## 29.0.8
2+
3+
* [swift] Adds validation errors for code that Swift FFI doesn't support:
4+
event channels, ProxyApis, sealed classes, data class fields that conflict with
5+
`NSObject` members, host API parameters named `wrappedError`, and
6+
`includeErrorClass: false`.
7+
* [swift] Fixes Swift FFI code that didn't compile for classes without fields,
8+
`@asyncCallback` methods, `@SwiftFunction` overloads, methods named `type`,
9+
parameters named `error`, and multiple FFI files in the same module.
10+
* [kotlin] Adds validation errors for code that Kotlin JNI doesn't support:
11+
event channels, ProxyApis, and sealed classes. Files that use Kotlin JNI
12+
previously generated method channel code for event channels and ProxyApis;
13+
move them to a separate pigeon file that doesn't use Kotlin JNI.
14+
* [dart] Adds a validation error for API method names that conflict with
15+
generated native interop members.
16+
* [dart] Fixes native interop code that didn't compile for JNI member names that
17+
JNIgen renames, parameter names that match generated local variables, lists or
18+
maps of ProxyApis, a `fileSpecificClassNameComponent` that isn't
19+
UpperCamelCase or is set only in `SwiftOptions`, and Dart test files.
20+
* [dart] Fixes a crash converting classes without fields with Swift FFI.
21+
* [dart] Fixes Swift FFI code that didn't compile for method names that end
22+
with a preposition, like `signIn`, or first parameter and field names that
23+
start with one, like `forKey`.
24+
125
## 29.0.7
226

327
* [kotlin] Fixes native interop calls failing with `ClassNotFoundException` in minified

‎packages/pigeon/example/native_interop_app/ios/Runner/NativeInteropExample.g.swift‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,7 @@ private func nilOrValue<T>(_ value: Any?) -> T? {
197197

198198
@objc class NativeInteropExamplePigeonInternalNull: NSObject {}
199199

200-
class _PigeonFfiCodec {
200+
private class _PigeonFfiCodec {
201201
static func readValue(value: NSObject?, type: String? = nil, type2: String? = nil) -> Any? {
202202
if NativeInteropExamplePigeonInternal.isNullish(value) {
203203
return nil
@@ -349,14 +349,15 @@ protocol NativeInteropExampleApi {
349349
}
350350
@objc func doSomething(wrappedError: PigeonError) {
351351
do {
352-
return try api!.doSomething()
352+
try api!.doSomething()
353+
return
353354
} catch let error as PigeonError {
354355
wrappedError.code = error.code
355356
wrappedError.message = error.message
356357
wrappedError.details = error.details
357358
} catch let error {
358359
wrappedError.code = "\(error)"
359-
wrappedError.message = "\(type(of: error))"
360+
wrappedError.message = "\(Swift.type(of: error))"
360361
wrappedError.details = "Stacktrace: \(Thread.callStackSymbols)"
361362
}
362363
return

‎packages/pigeon/example/native_interop_app/lib/src/native_interop_example.g.dart‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -482,9 +482,9 @@ class NativeInteropExampleApiForNativeInterop {
482482
if (_jniApi != null) {
483483
return _jniApi.doSomething();
484484
} else if (_ffiApi != null) {
485-
final error = ffi_bridge.PigeonError();
486-
_ffiApi.doSomethingWithWrappedError(error);
487-
_throwIfFfiError(error);
485+
final pigeonVar_error = ffi_bridge.PigeonError();
486+
_ffiApi.doSomethingWithWrappedError(pigeonVar_error);
487+
_throwIfFfiError(pigeonVar_error);
488488
return;
489489
} else {
490490
throw Exception('No JNI or FFI api available');

‎packages/pigeon/lib/src/dart/dart_generator.dart‎

Lines changed: 411 additions & 88 deletions
Large diffs are not rendered by default.

‎packages/pigeon/lib/src/generator_tools.dart‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ import 'generator.dart';
1616
/// The current version of pigeon.
1717
///
1818
/// This must match the version in pubspec.yaml.
19-
const String pigeonVersion = '29.0.7';
19+
const String pigeonVersion = '29.0.8';
2020

2121
/// Default plugin package name.
2222
const String defaultPluginPackageName = 'dev.flutter.pigeon';

‎packages/pigeon/lib/src/kotlin/kotlin_generator.dart‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,10 @@ class KotlinOptions {
7777
///
7878
/// This should only ever be set to false if you have another generated
7979
/// Kotlin file in the same directory.
80+
///
81+
/// With [useJni], the other file must also use the same [package] and
82+
/// [errorClassName], because the JNIgen bindings expect the error class at
83+
/// `<package>.<errorClassName>`.
8084
final bool includeErrorClass;
8185

8286
/// A String to augment class names to avoid cross file collisions.

‎packages/pigeon/lib/src/pigeon_lib_internal.dart‎

Lines changed: 224 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -131,20 +131,7 @@ class InternalPigeonOptions {
131131
useFfi: options.swiftOptions?.useFfi ?? false,
132132
ffiErrorClassName: options.swiftOptions?.errorClassName ?? 'PigeonError',
133133
jniErrorClassName: options.kotlinOptions?.errorClassName ?? 'FlutterError',
134-
fileSpecificClassNameComponent:
135-
options.fileSpecificClassNameComponent ??
136-
(options.swiftOptions?.useFfi ?? false
137-
? options.swiftOptions?.fileSpecificClassNameComponent ??
138-
(options.swiftOutPaths?.firstOrNull == null
139-
? null
140-
: deduceClassNameComponent(options.swiftOutPaths!.first))
141-
: null) ??
142-
(options.kotlinOptions?.useJni ?? false
143-
? options.kotlinOptions?.fileSpecificClassNameComponent ??
144-
(options.kotlinOut == null
145-
? null
146-
: deduceClassNameComponent(options.kotlinOut))
147-
: null),
134+
fileSpecificClassNameComponent: _dartNativeInteropClassNameComponent(options),
148135
),
149136
copyrightHeader = options.copyrightHeader != null
150137
? _lineReader(path.posix.join(options.basePath ?? '', options.copyrightHeader))
@@ -218,6 +205,30 @@ Iterable<String> _lineReader(String path) sync* {
218205
}
219206
}
220207

208+
/// The class name component that the Dart generator uses to reference the
209+
/// per-file helper classes declared by the native interop generators.
210+
///
211+
/// This must match the derivation in `InternalSwiftOptions.fromSwiftOptions`
212+
/// and `InternalKotlinOptions.fromKotlinOptions`, including the precedence of
213+
/// the language-specific option and the conversion to UpperCamelCase.
214+
String? _dartNativeInteropClassNameComponent(PigeonOptions options) {
215+
final String? component;
216+
if (options.swiftOptions?.useFfi ?? false) {
217+
component =
218+
options.swiftOptions?.fileSpecificClassNameComponent ??
219+
options.fileSpecificClassNameComponent ??
220+
deduceClassNameComponent(options.swiftOutPaths?.firstOrNull);
221+
} else if (options.kotlinOptions?.useJni ?? false) {
222+
component =
223+
options.kotlinOptions?.fileSpecificClassNameComponent ??
224+
options.fileSpecificClassNameComponent ??
225+
deduceClassNameComponent(options.kotlinOut);
226+
} else {
227+
component = options.fileSpecificClassNameComponent;
228+
}
229+
return component == null ? null : toUpperCamelCase(component);
230+
}
231+
221232
File _getFile(String output, {String basePath = ''}) {
222233
final file = File(path.posix.join(basePath, output));
223234
file.createSync(recursive: true);
@@ -325,6 +336,167 @@ void _errorOnJniPropertyCollisions(List<Error> errors, Root root) {
325336
}
326337
}
327338

339+
/// Names that Swift FFI data class fields can't use.
340+
///
341+
/// FFI bridge classes subclass `NSObject`, so a field with one of these names
342+
/// either fails to compile in Swift, because it conflicts with an `NSObject`
343+
/// member, or is renamed by FFIgen, because it conflicts with a member of the
344+
/// Dart `NSObject` bindings.
345+
const Set<String> _swiftFfiReservedFieldNames = <String>{
346+
// Swift compile errors.
347+
'attributeKeys',
348+
'autoContentAccessingProxy',
349+
'autorelease',
350+
'classCode',
351+
'classDescription',
352+
'classForArchiver',
353+
'classForCoder',
354+
'classForKeyedArchiver',
355+
'classForPortCoder',
356+
'className',
357+
'copy',
358+
'dealloc',
359+
'debugDescription',
360+
'description',
361+
'finalize',
362+
'hash',
363+
'hashValue',
364+
'isProxy',
365+
'mutableCopy',
366+
'objectSpecifier',
367+
'observationInfo',
368+
'release',
369+
'retain',
370+
'retainCount',
371+
'scriptingProperties',
372+
'superclass',
373+
'toManyRelationshipKeys',
374+
'toOneRelationshipKeys',
375+
'zone',
376+
// FFIgen renames.
377+
'alloc',
378+
'allocWithZone',
379+
'conformsToProtocol',
380+
'copyWithZone',
381+
'doesNotRecognizeSelector',
382+
'forwardInvocation',
383+
'forwardingTargetForSelector',
384+
'initialize',
385+
'instanceMethodForSelector',
386+
'instanceMethodSignatureForSelector',
387+
'instancesRespondToSelector',
388+
'isA',
389+
'isEqual',
390+
'isKindOfClass',
391+
'isMemberOfClass',
392+
'isSubclassOfClass',
393+
'load',
394+
'methodSignatureForSelector',
395+
'mutableCopyWithZone',
396+
'performSelector',
397+
'ref',
398+
'resolveClassMethod',
399+
'resolveInstanceMethod',
400+
};
401+
402+
const String _moveToNonFfiFileSuggestion =
403+
"Move them to a separate pigeon file that doesn't use Swift FFI (`SwiftOptions.useFfi` or `--swift_use_ffi`).";
404+
405+
void _errorOnSwiftFfiUnsupportedFeatures(
406+
List<Error> errors,
407+
InternalSwiftOptions swiftOptions,
408+
Root root,
409+
) {
410+
for (final Api api in root.apis) {
411+
if (api is AstEventChannelApi) {
412+
errors.add(
413+
Error(
414+
message:
415+
'Swift FFI does not support event channels yet (in API "${api.name}"). $_moveToNonFfiFileSuggestion',
416+
),
417+
);
418+
} else if (api is AstProxyApi) {
419+
errors.add(
420+
Error(
421+
message:
422+
'Swift FFI does not support ProxyApis yet (in API "${api.name}"). $_moveToNonFfiFileSuggestion',
423+
),
424+
);
425+
} else if (api is AstHostApi) {
426+
for (final Method method in api.methods) {
427+
if (method.parameters.any((Parameter parameter) => parameter.name == 'wrappedError')) {
428+
errors.add(
429+
Error(
430+
message:
431+
'Parameter "wrappedError" in method "${method.name}" in API "${api.name}" conflicts with the error parameter that Swift FFI adds to host API methods. Rename the parameter.',
432+
),
433+
);
434+
}
435+
}
436+
}
437+
}
438+
for (final Class classDefinition in root.classes) {
439+
if (classDefinition.isSealed) {
440+
errors.add(
441+
Error(
442+
message:
443+
'Swift FFI does not support sealed classes yet (class "${classDefinition.name}"). $_moveToNonFfiFileSuggestion',
444+
),
445+
);
446+
}
447+
for (final NamedType field in classDefinition.fields) {
448+
// `description` has its own error for all Swift generation.
449+
if (field.name != 'description' && _swiftFfiReservedFieldNames.contains(field.name)) {
450+
errors.add(
451+
Error(
452+
message:
453+
'Field "${field.name}" is not allowed in class "${classDefinition.name}" with Swift FFI because it conflicts with a member of NSObject, which FFI bridge classes extend. Rename the field.',
454+
),
455+
);
456+
}
457+
}
458+
}
459+
if (!swiftOptions.includeErrorClass) {
460+
errors.add(
461+
Error(
462+
message:
463+
'Swift FFI requires `includeErrorClass: true`, because the FFI bindings are generated from this file alone, so the error class "${swiftOptions.errorClassName ?? 'PigeonError'}" must be declared in it. To use Swift FFI in more than one pigeon file in the same module, give each file a different `errorClassName` instead.',
464+
),
465+
);
466+
}
467+
}
468+
469+
const String _moveToNonJniFileSuggestion =
470+
"Move them to a separate pigeon file that doesn't use Kotlin JNI (`KotlinOptions.useJni` or `--kotlin_use_jni`).";
471+
472+
void _errorOnKotlinJniUnsupportedFeatures(List<Error> errors, Root root) {
473+
for (final Api api in root.apis) {
474+
if (api is AstEventChannelApi) {
475+
errors.add(
476+
Error(
477+
message:
478+
'Kotlin JNI does not support event channels yet (in API "${api.name}"). $_moveToNonJniFileSuggestion',
479+
),
480+
);
481+
} else if (api is AstProxyApi) {
482+
errors.add(
483+
Error(
484+
message:
485+
'Kotlin JNI does not support ProxyApis yet (in API "${api.name}"). $_moveToNonJniFileSuggestion',
486+
),
487+
);
488+
}
489+
}
490+
for (final Class classDefinition in root.classes.where((Class c) => c.isSealed)) {
491+
errors.add(
492+
Error(
493+
message:
494+
'Kotlin JNI does not support sealed classes yet (class "${classDefinition.name}"). $_moveToNonJniFileSuggestion',
495+
),
496+
);
497+
}
498+
}
499+
328500
/// A [GeneratorAdapter] that generates the AST.
329501
class AstGeneratorAdapter implements GeneratorAdapter {
330502
/// Constructor for [AstGeneratorAdapter].
@@ -372,7 +544,42 @@ class DartGeneratorAdapter implements GeneratorAdapter {
372544
_openSink(options.dartOptions?.dartOut, basePath: options.basePath ?? '');
373545

374546
@override
375-
List<Error> validate(InternalPigeonOptions options, Root root) => <Error>[];
547+
List<Error> validate(InternalPigeonOptions options, Root root) {
548+
final InternalDartOptions? dartOptions = options.dartOptions;
549+
if (dartOptions == null || !usesNativeInterop(dartOptions)) {
550+
return <Error>[];
551+
}
552+
return _errorOnNativeInteropReservedMethodNames(root, useJni: dartOptions.useJni);
553+
}
554+
}
555+
556+
/// Returns errors for API methods whose names conflict with members of the
557+
/// Dart classes that native interop generates for the API.
558+
List<Error> _errorOnNativeInteropReservedMethodNames(Root root, {required bool useJni}) {
559+
final errors = <Error>[];
560+
for (final Api api in root.apis) {
561+
final Set<String> reservedNames = switch (api) {
562+
AstHostApi() => const <String>{'createWithNativeInteropApi', 'getInstance'},
563+
// The JNI registrar implements the JNIgen interface for the API, so it
564+
// shares a namespace with the API's methods.
565+
AstFlutterApi() => <String>{
566+
'implement',
567+
if (useJni) ...<String>{'dartApi', 'register'},
568+
},
569+
_ => const <String>{},
570+
};
571+
for (final Method method in api.methods) {
572+
if (reservedNames.contains(method.name)) {
573+
errors.add(
574+
Error(
575+
message:
576+
'Method name "${method.name}" in API "${api.name}" conflicts with a member of the Dart code that native interop generates for the API. Rename the method.',
577+
),
578+
);
579+
}
580+
}
581+
}
582+
return errors;
376583
}
377584

378585
/// A [GeneratorAdapter] that generates Dart test source code.
@@ -555,6 +762,7 @@ class SwiftGeneratorAdapter implements GeneratorAdapter {
555762
}
556763
if (options.swiftOptions?.useFfi ?? false) {
557764
_errorOnTaskQueueInNativeInterop(errors, 'Swift FFI', root);
765+
_errorOnSwiftFfiUnsupportedFeatures(errors, options.swiftOptions!, root);
558766
}
559767
return errors;
560768
}
@@ -749,6 +957,7 @@ class KotlinGeneratorAdapter implements GeneratorAdapter {
749957
if (options.kotlinOptions?.useJni ?? false) {
750958
_errorOnTaskQueueInNativeInterop(errors, 'Kotlin JNI', root);
751959
_errorOnJniPropertyCollisions(errors, root);
960+
_errorOnKotlinJniUnsupportedFeatures(errors, root);
752961
}
753962
return errors;
754963
}

0 commit comments

Comments
 (0)