Skip to content

Add iter()/aiter() for filter-based lazy iteration #489

Description

@tylerhutcherson

Summary

Add lazy filter-based ID/key iteration APIs for index maintenance and agent pipelines.

Level: Intermediate

Current State

  • paginate() yields full document records, not IDs/keys only.
  • scan_by_pattern() works on raw Redis key patterns and returns a materialized list.
  • No index filter-expression based generator for IDs/keys.

Proposed Change

  • Add lazy iterators over keys included in an index in Redis optionally applying some filtering:
    • iter(filter_expression...)
    • aiter(filter_expression,...)
  • Yield as a generator of said keys?

Definition of Done

  • Iteration is lazy (streaming batches, no full list required).
  • Works for selective and wildcard filter expressions.
  • Integration tests verify complete and deterministic coverage.

Out of Scope

  • Arbitrary cursor checkpoints or resumable iterators.

Activity

  1. changed the title [-]Add iter_ids()/aiter_ids() for filter-based lazy iteration[/-] [+]Add iter()/aiter() for filter-based lazy iteration[/+] on Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions