Skip to content

Comments

feat(napi-derive): add #[napi(async_iterator)] macro attribute#3072

Merged
Brooooooklyn merged 4 commits intomainfrom
async-iterator
Dec 28, 2025
Merged

feat(napi-derive): add #[napi(async_iterator)] macro attribute#3072
Brooooooklyn merged 4 commits intomainfrom
async-iterator

Conversation

@Brooooooklyn
Copy link
Member

@Brooooooklyn Brooooooklyn commented Dec 28, 2025

Add macro support for #[napi(async_iterator)] to mirror the existing
#[napi(iterator)] API, leveraging the AsyncGenerator trait and
create_async_iterator() runtime function.

Key changes:

  • Add async_iterator attribute to attrs.rs
  • Add async iterator fields to AST (NapiStruct, NapiClass, NapiImpl, NapiFn)
  • Parse async_iterator and extract AsyncGenerator trait types
  • Add mutual exclusivity check with #[napi(iterator)]
  • Generate [Symbol.asyncIterator]() method in TypeScript
  • Add construct_async_generator and async_generator_factory methods
  • Export create_async_iterator in __private module

TypeScript generates proper async iterable type:

export declare class AsyncFib {
  [Symbol.asyncIterator](): AsyncGenerator<number, void, number | undefined>
}

TNext is T | undefined to allow for await...of loops which call
next() with no arguments.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.5 [email protected]


Note

Adds first-class async iterator support parallel to sync iterators.

  • Parser/AST: new #[napi(async_iterator)] attribute, mutual exclusivity with iterator, and new flags/types on NapiFn, NapiStruct, NapiClass, NapiImpl (async iterator Yield/Next/Return)
  • Codegen: construct/factory paths for async generators (construct_async_generator, async_generator_factory); auto-attach async iterator on instances
  • Runtime: export create_async_iterator; ensure return() sets done: true; plumb async generator trait under tokio_rt
  • Typegen: emit [Symbol.asyncIterator](): AsyncGenerator<Y, R, N> with TNext | undefined for for await...of
  • Examples/exports/tests: add AsyncFib, DelayedCounter, AsyncDataSource with comprehensive async-iterator tests and module re-exports
  • Chore: bump TS target to ES2024

Written by Cursor Bugbot for commit c5b3706. This will update automatically on new commits. Configure here.

Summary by CodeRabbit

  • New Features

    • Async iterators/generators added: Symbol.asyncIterator / AsyncGenerator typings, runtime helpers (tokio_rt-gated), and module exports for async generators.
  • Examples

    • Added AsyncFib, DelayedCounter, AsyncDataSource with constructors/factories and async-iteration behavior.
  • Tests

    • Added comprehensive async-iterator tests (for-await-of, next/return/throw, concurrency, delays).
  • Chores

    • TypeScript target updated to ES2024.

✏️ Tip: You can customize this high-level summary in your review settings.

Add macro support for `#[napi(async_iterator)]` to mirror the existing
`#[napi(iterator)]` API, leveraging the `AsyncGenerator` trait and
`create_async_iterator()` runtime function.

Key changes:
- Add `async_iterator` attribute to attrs.rs
- Add async iterator fields to AST (NapiStruct, NapiClass, NapiImpl, NapiFn)
- Parse `async_iterator` and extract `AsyncGenerator` trait types
- Add mutual exclusivity check with `#[napi(iterator)]`
- Generate `[Symbol.asyncIterator]()` method in TypeScript
- Add `construct_async_generator` and `async_generator_factory` methods
- Export `create_async_iterator` in `__private` module

TypeScript generates proper async iterable type:
```typescript
export declare class AsyncFib {
  [Symbol.asyncIterator](): AsyncGenerator<number, void, number | undefined>
}
```

TNext is `T | undefined` to allow `for await...of` loops which call
`next()` with no arguments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Copilot AI review requested due to automatic review settings December 28, 2025 06:18
@coderabbitai
Copy link

coderabbitai bot commented Dec 28, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

📝 Walkthrough

Walkthrough

Adds end-to-end async generator/async-iterator support: AST flags/types, macro parsing for #[napi(async_iterator)], codegen paths for async-generator constructors/factories, runtime helpers and AsyncGenerator runtime module (tokio_rt-gated), TypeScript typings, and three async-iterable example classes with tests and exports.

Changes

Cohort / File(s) Summary
AST Structure Definitions
crates/backend/src/ast.rs
Added fields: parent_is_async_generator: bool (NapiFn), is_async_generator: bool (NapiStruct), implement_async_iterator: bool (NapiClass), and async_iterator_yield_type, async_iterator_next_type, async_iterator_return_type: Option<Type> (NapiImpl).
Macro Parser & Attributes
crates/macro/src/parser/attrs.rs, crates/macro/src/parser/mod.rs
Added AsyncIterator(Span) attr; changed generator tracking to (is_sync_generator, is_async_generator); added implement_async_iterator and mutual-exclusivity guard vs iterator; propagate parent async flag to NapiFn and async types to NapiImpl.
Code Generation Logic
crates/backend/src/codegen/fn.rs, crates/backend/src/codegen/struct.rs
Route constructor/generator/factory paths to async variants (construct_async_generator, async_generator_factory) when parent is async generator; added gen_async_iterator_property and wired async iterator creation into ToNapiValue / reference flows.
Type Definition Generation
crates/backend/src/typegen/struct.rs
Emit JSDoc for async iterable protocol when is_async_generator is true; generate [Symbol.asyncIterator](): AsyncGenerator<Yield, Return, Next> signatures from async_iterator_*_type (Next may include undefined, Return defaults to void).
Runtime Support
crates/napi/src/bindgen_runtime/callback_info.rs, crates/napi/src/bindgen_runtime/mod.rs, crates/napi/src/bindgen_runtime/async_iterator.rs, crates/napi/src/lib.rs
Added construct_async_generator and async_generator_factory (tokio_rt-gated); added async_iterator module and AsyncGenerator re-export; exposed create_async_iterator via __private when tokio_rt feature enabled; adjusted generator_return to always mark done=true.
Examples & Tests
examples/napi/src/generator.rs, examples/napi/__tests__/*, examples/napi/index.*, examples/napi/example.wasi*.{js,cjs}, examples/napi/__tests__/__snapshots__/*
Implemented AsyncFib, DelayedCounter, AsyncDataSource with AsyncGenerator impls and async next() futures (tokio sleep where applicable); added tests exercising for-await-of, next/return/throw semantics, concurrent iteration; exported new classes through bindings and updated snapshots.
Configuration
tsconfig.json
Bumped TS target from ES2022 to ES2024.

Sequence Diagram(s)

sequenceDiagram
    participant Macro as Macro Parser
    participant AST as AST
    participant Codegen as Code Generator
    participant Callback as CallbackInfo
    participant Runtime as AsyncIterator Runtime
    participant JS as JavaScript

    Note over Macro,AST: Compile-time parsing & AST population
    Macro->>AST: parse #[napi(async_iterator)] and impl async_iterator types
    AST->>Codegen: emit Symbol.asyncIterator sig & gen_async_iterator_property()
    Note over Codegen,Callback: emit runtime constructor/factory calls
    JS->>Callback: call constructor/factory
    Callback->>Callback: construct_async_generator / async_generator_factory
    Callback->>Runtime: create_async_iterator(instance)
    Runtime->>JS: return wrapped instance with Symbol.asyncIterator
    JS->>Runtime: iterator.next() -> returns Promise
    Runtime->>Runtime: poll associated Future (async next)
    Runtime-->>JS: Promise resolves with { value, done }
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 I dug a burrow in async night,

Yields and promises take flight.
For-await I dance, soft and fleet,
New iterators hop to the beat.
Carrots of code — a tasty async treat!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 29.17% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically describes the main change: adding a new #[napi(async_iterator)] macro attribute. It accurately reflects the core feature introduced across all modified files.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch async-iterator

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 07daf29 and c5b3706.

📒 Files selected for processing (1)
  • crates/napi/src/bindgen_runtime/async_iterator.rs
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: napi-rs/napi-rs PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T09:31:23.877Z
Learning: Applies to /crates/**/*.rs : Use `#[napi]` attribute on impl blocks to define classes exported to JavaScript/TypeScript
Learnt from: CR
Repo: napi-rs/napi-rs PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T09:31:23.877Z
Learning: Run `yarn workspace examples/napi build` before testing to ensure code generation reflects current Rust changes
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (41)
  • GitHub Check: x86_64-unknown-linux-gnu - node@22 - toolchain@ stable
  • GitHub Check: x86_64-unknown-linux-gnu - node@24 - toolchain@ 1.88.0
  • GitHub Check: i686-pc-windows-msvc - node@22 - toolchain@ stable
  • GitHub Check: x86_64-unknown-linux-gnu - node@24 - toolchain@ stable
  • GitHub Check: x86_64-pc-windows-msvc - node@20 - toolchain@ stable
  • GitHub Check: i686-pc-windows-msvc - node@24 - toolchain@ stable
  • GitHub Check: x86_64-pc-windows-msvc - node@24 - toolchain@ stable
  • GitHub Check: aarch64-apple-darwin - node@24 - toolchain@ stable
  • GitHub Check: aarch64-apple-darwin - node@20 - toolchain@ stable
  • GitHub Check: x86_64-unknown-linux-gnu - node@20 - toolchain@ stable
  • GitHub Check: aarch64-pc-windows-msvc - node@22 - toolchain@ stable
  • GitHub Check: aarch64-pc-windows-msvc - node@24 - toolchain@ stable
  • GitHub Check: x86_64-unknown-linux-gnu - node@22 - toolchain@ 1.88.0
  • GitHub Check: aarch64-apple-darwin - node@22 - toolchain@ stable
  • GitHub Check: x86_64-pc-windows-msvc - node@22 - toolchain@ stable
  • GitHub Check: Build only test - riscv64gc-unknown-linux-gnu
  • GitHub Check: Build only test - aarch64-linux-android
  • GitHub Check: build - x86_64-unknown-linux-gnu
  • GitHub Check: Build only test - loongarch64-unknown-linux-gnu
  • GitHub Check: Test freebsd target
  • GitHub Check: Build only test - aarch64-unknown-linux-ohos
  • GitHub Check: Test node wasi target (--cfg tokio_unstable)
  • GitHub Check: build-and-test-msys2 (MINGW64)
  • GitHub Check: Build only test - armv7-linux-androideabi
  • GitHub Check: build - powerpc64le-unknown-linux-gnu
  • GitHub Check: build - s390x-unknown-linux-gnu
  • GitHub Check: build - aarch64-unknown-linux-gnu
  • GitHub Check: stable - armv7-unknown-linux-gnueabihf - node@22
  • GitHub Check: Test node wasi target
  • GitHub Check: build-and-test-msys2 (CLANG64)
  • GitHub Check: build - aarch64-unknown-linux-musl
  • GitHub Check: build-and-test-msys2 (UCRT64)
  • GitHub Check: build - x86_64-unknown-linux-musl
  • GitHub Check: Memory leak detect job
  • GitHub Check: ASAN - ubuntu-24.04
  • GitHub Check: ASAN - windows-latest
  • GitHub Check: Zig-Cross-Compile-On-Linux (aarch64-unknown-linux-musl)
  • GitHub Check: Zig-Cross-Compile-On-Linux (armv7-unknown-linux-musleabihf)
  • GitHub Check: Zig-Cross-Compile-On-Linux (x86_64-unknown-linux-musl)
  • GitHub Check: Zig-Cross-Compile-On-Linux (aarch64-apple-darwin)
  • GitHub Check: Cursor Bugbot
🔇 Additional comments (1)
crates/napi/src/bindgen_runtime/async_iterator.rs (1)

341-349: LGTM! Correct protocol compliance fix.

The change correctly ensures that AsyncGenerator.return() always resolves to { done: true, value } per the ECMAScript async iterator protocol. When return() is called on an async generator, the iteration is complete regardless of whether a value is present, so done must always be true.

The previous implementation that could set done: false when complete() returned Some(value) was non-compliant with the specification.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Member Author


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • ready-to-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

if let Some(v) = value {
obj.set("value", v)?;
obj.set("done", false)?;
Ok(obj)

P1 Badge AsyncGenerator.return() reports done=false when yielding a value

In the async generator return path the resolved promise sets done to false whenever complete() returns a value (value is Some), only marking completion when complete() returns None. According to the async iterator protocol, return() must always resolve to { done: true, value }, even if a final value is provided. With the current logic, any AsyncGenerator override of complete() that wants to return a final value will signal done: false, causing consumers (including for await...of) to keep iterating and invoke next() again, contrary to spec and likely to produce duplicate work.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for #[napi(async_iterator)] macro attribute, enabling Rust structs to implement JavaScript's async iterable protocol. This mirrors the existing #[napi(iterator)] functionality but for asynchronous iteration patterns.

Key Changes:

  • Added AsyncGenerator trait with next(), complete(), and catch() methods returning futures
  • Implemented macro parsing and code generation for #[napi(async_iterator)] attribute with mutual exclusivity check against #[napi(iterator)]
  • Generated TypeScript definitions with [Symbol.asyncIterator](): AsyncGenerator<TYield, TReturn, TNext> signature where TNext includes undefined for for await...of compatibility

Reviewed changes

Copilot reviewed 16 out of 18 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
tsconfig.json Updated TypeScript target from ES2022 to ES2024 for async generator support
examples/napi/src/generator.rs Added three example async iterator implementations: AsyncFib (basic), DelayedCounter (with delays), and AsyncDataSource (factory pattern)
examples/napi/index.d.cts Generated TypeScript declarations for async iterator classes with JSDoc explaining async iterable protocol
examples/napi/index.cjs Added exports for AsyncDataSource, AsyncFib, and DelayedCounter classes
examples/napi/example.wasi.cjs Added WASI exports for new async iterator classes
examples/napi/example.wasi-browser.js Added browser WASI exports for new async iterator classes
examples/napi/tests/generator.spec.ts Added comprehensive test coverage for async generators including for-await-of loops, next(), return(), completion, and concurrency tests
examples/napi/tests/snapshots/values.spec.ts.snap Binary snapshot file updated with new TypeScript definitions
examples/napi/tests/snapshots/values.spec.ts.md Markdown snapshot updated showing generated TypeScript for async iterator classes
crates/napi/src/lib.rs Exported create_async_iterator in __private module with tokio_rt feature gate
crates/napi/src/bindgen_runtime/mod.rs Exported AsyncGenerator trait with tokio_rt feature gate
crates/napi/src/bindgen_runtime/callback_info.rs Added construct_async_generator and async_generator_factory methods mirroring sync generator equivalents
crates/macro/src/parser/mod.rs Added parsing logic for async_iterator attribute, mutual exclusivity validation, and tracking of async generator types
crates/macro/src/parser/attrs.rs Added async_iterator attribute definition to the macro attribute system
crates/backend/src/typegen/struct.rs Added TypeScript type generation for async iterators with proper TNext handling (includes undefined)
crates/backend/src/codegen/struct.rs Added gen_async_iterator_property for calling create_async_iterator in generated code
crates/backend/src/codegen/fn.rs Added async generator handling in constructor and factory method code generation
crates/backend/src/ast.rs Added async_iterator fields to NapiFn, NapiStruct, NapiClass, and NapiImpl AST nodes

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Brooooooklyn and others added 3 commits December 28, 2025 14:55
…rrency test

WASI environments in CI can be slower, causing the 150ms tolerance to fail.
Increase to 300ms which still validates concurrency while accommodating
slower CI environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Improve comments in AsyncFib example to clarify the pattern
- Expand comment explaining why create_async_iterator is not unsafe
- Improve error message for iterator/async_iterator mutual exclusivity
- Add throw() test for async generators
- Add idempotency check for completed async generator state
- Make skip messages consistent (mention tokio_rt feature)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Per the async iterator protocol, return() must ALWAYS resolve to
{ done: true, value } even when complete() returns a final value.
The previous implementation incorrectly set done: false when
complete() returned Some(value).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@Brooooooklyn
Copy link
Member Author

Fixed the bug reported by Codex Review: AsyncGenerator.return() now always sets done: true per the async iterator protocol, even when complete() returns a final value.

@Brooooooklyn Brooooooklyn merged commit 76d06b3 into main Dec 28, 2025
135 of 136 checks passed
@Brooooooklyn Brooooooklyn deleted the async-iterator branch December 28, 2025 07:56
@github-actions github-actions bot mentioned this pull request Dec 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant