[pigeon] Native interop is slower than channels for data classes and collections #193195

Follow-up to #182230.

Summary

Pigeon's native interop mode (useJni / useFfi) moves a data class as a graph of JVM or Objective-C objects. Every non-scalar field and every collection element costs at least one Dart→native call and one tracked native reference, all on the calling thread (the UI thread for the root isolate). Channels encode the payload in memory and cross the boundary once.

So the trigger is node count, not field count. A class with 30 non-null scalar fields is fast over native interop. A List<PlatformLatLng> with 1,000 elements, or a single class with many nullable, string, list or map fields, is not.

Past about 10 nodes, native interop is slower than channels on both platforms:

Small calls are where native interop shines, and any fix has to keep them that fast.

Root causes

1. One boundary crossing and one tracked reference per node. This is the core problem. Every other cause is a multiplier on top of it.
2. Every JNI type check is an uncached class lookup. isA calls JClass.forName, which on Android is a ClassLoader.loadClass upcall plus a global ref, and as() repeats it. The generated readValue tries types in schema order, so reading data class k costs 11 + k lookups (generated code). Measured: 0.66–0.72 µs uncached, against 0.03 µs on a cached class. In a 1,000-element list read, lookups are ~7.9 of the 8.7 µs per element.
3. Static types are thrown away. Typed fields and returns (List<int>, Map<String, String>, typed data) still go through the dynamic readValue, with 35 (JNI) and 32 (FFI) is List<…>/is Map<…> branches tried in order.
4. Hidden calls per element. JList.asDart()[i] calls size() on every index, maps cost three calls per entry, and FFI loops re-read count.
5. Every reference is global and finalized. package:jni promotes returned objects to global refs, each with a calloc and two finalizable handles.
6. Smaller items. Scalars inside collections are double-boxed on Apple. Kotlin ofRaw clones values() per enum. Typed data gets 3 copies plus a zeroing pass on FFI.

Proposal: a hybrid transport

Every API call makes exactly one native call. What crosses directly and what is serialized depends only on the static parameter type, never on class size:

Key decisions, each with its main reason:

• Wire format: the existing StandardMessageCodec format. The Dart codec is already in the generated file, Kotlin's only needs re-enabling in JNI mode (kotlin_generator.dart), and native interop then behaves exactly like its channel fallback. The format is internal, because Dart and native code are always generated together.
• Swift: a new Foundation-only codec. swiftgen compiles the FFI Swift file against the bare SDK, so FlutterStandardReader/Writer aren't available (swift_generator.dart). It is fileprivate, so it adds no Objective-C symbols. Golden byte vectors from Dart keep it honest.
• Owned buffers, one copy per crossing. Borrowed buffers are unsafe: an @objc async Swift method saw the caller's reused buffer in 498–500 of 500 trials. Flutter's ReadBuffer also returns typed data as views, so decoding from native-owned memory leaves dangling fields. The copies cost 2–3% of a round trip.
• One internal pigeon_ entry point per method, not one dispatch(id, buffer) per API. Single dispatch would shrink bindings further (−98% vs −56% on JNI), but scalar calls would go through a buffer and get ~50× slower (2.1 µs vs 0.04 µs).
• Errors keep today's paths (Kotlin exceptions, the FFI wrappedError), with details encoded by the codec, so structured details work on both platforms.
• Async and FlutterApi use the same buffers. Owned buffers make suspension safe, and an @async method can hop to a background queue before decoding.

What it buys

• O(1) crossings and O(1) tracked references per call. Scalar calls keep exactly today's direct call.
• Data classes leave the jnigen/ffigen inputs. Bindings shrink by 56% on JNI and 21% on FFI, and adding a field no longer needs a binding regeneration or keep rules.
• The object-graph codecs, the Swift …Bridge classes, NumberWrapper, PigeonInternalNull and PigeonTypedData are deleted.

Prototype results (Android, release, against today and channels):

The hybrid beat channels at every size measured, because it skips their ~40 µs fixed cost. The one regression is tiny class arguments (+1.4 µs). Apple isn't prototyped yet; that is the main open risk, since macOS channels took 4× as long as Android for the same payload.

Options rejected: tuning the object graph (still O(nodes), and writes barely improve), a field-count threshold (a cliff when a class grows), serializing every argument (slow scalar calls), columnar packing and C structs (reinvent serialization, or don't work on Android), lazy proxies (break value semantics), and docs only.

Plan

0. Measure. A proper benchmark harness (warm-up, ≥200 samples, release mode, real plugin shapes, debug counters for crossings and refs), with baselines on a low-end Android phone and iOS.
1. Fixes that survive the redesign (patch releases): R8 keep rules plus a minified build in CI; reject FFI files with event channels or ProxyApis; stop binding the HostApi interface; O(1) ofRaw; static converters for top-level typed returns; and one stopgap, caching a JClass per type in the generated JNI codec (estimated 8.7 → ~1.2 ms for the 1,000-element read).
2. Hybrid on Android behind a per-file flag (minor release).
3. Hybrid on Apple behind the flag (minor release), with the Swift codec benchmarked on its own.
4. Flip the default and delete the object-graph path (major release, since generated public types are removed).

Budgets for the flip: never slower than channels for any shape up to N = 10,000; scalar calls within 10% of today; updateMarkers with 1,000 markers under 8 ms of UI-thread time on a low-end phone; tiny-class calls at most 3 µs slower.

Workarounds until then

1. Keep bulk-data APIs in a separate pigeon file without useJni/useFfi.
2. Send bulk numbers as typed data, e.g. a Float64List of lat/lng pairs instead of List<PlatformLatLng>.
3. Flatten hot tiny classes into scalar parameters.
4. Send deltas, not snapshots.
5. Avoid returning large collections or data classes over JNI; reads are the slowest path.
6. Add R8 keep rules for the generated classes (and kotlin.Unit, kotlin.Result, kotlin.Result$Failure, kotlin.coroutines.Continuation, kotlin.coroutines.intrinsics.CoroutineSingletons, kotlin.coroutines.intrinsics.IntrinsicsKt), and test a minified release build.
7. Measure in profile or release mode; debug builds exaggerate all of this.

Open questions

• Which low-end Android device should be the reference for the budgets?
• Can a Foundation-only Swift codec match the Kotlin codec's speed? Phase 3 answers this; if not, profile before changing the format.