Skip to content

Add delete_where() API for bulk index deletion by filter #488

Description

@tylerhutcherson

Summary

Add delete_where() / adelete_where() to delete index documents by filter expression, and refactor clear() to share the same internals.

Level: Intermediate

Current State

  • clear() deletes all documents via batched FilterQuery("*") flow.
  • drop_documents() / drop_keys() only delete known IDs/keys.
  • There is no public API to delete a filtered subset of index documents.

Proposed Change

  • Add:
    • delete_where(filter_expression, batch_size=500) -> int
    • adelete_where(filter_expression, batch_size=500) -> int
  • Refactor both clear() and delete_where() to use shared private internals:
    • common batch-ID fetch loop
    • existing _delete_batch() and async equivalent as delete primitive
  • Implement clear() as delete_where(FilterExpression("*")).

Definition of Done

  • Filtered bulk deletion works on standalone and cluster clients.
  • clear() delegates to the shared deletion internals (no duplicated batch loop).
  • Integration tests cover selective match, no-op match, and full-clear behavior.
  • Docs include one usage example.

Out of Scope

  • Hard-delete audit logging or soft-delete support.

Activity

  1. Aryan-Pardeshi commented on Aug 10, 2026

    @Aryan-Pardeshi

    Looks like this was already addressed by drop_by_filter() / adrop_by_filter() on main (index.py:1306 sync, 2638 async) — deletes by filter expression, with batch_size, dry_run, allow_all, and on_progress, and uses a runaway backstop plus offset-0 re-querying to stay correct as the match set shrinks. Looks like this issue can be closed, unless there's a gap between what shipped and what was originally asked for that I'm missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions