Skip to content

feature(ingestion): conversion events buffer consumer#9429

Closed
yakkomajuri wants to merge 2 commits intoevents-bufferfrom
events-buffer-consumer
Closed

feature(ingestion): conversion events buffer consumer#9429
yakkomajuri wants to merge 2 commits intoevents-bufferfrom
events-buffer-consumer

Conversation

@yakkomajuri
Copy link
Copy Markdown
Contributor

Problem

#9182

Changes

Still WIP!

Follow up to #9427.

Implements the consumer for the buffer.

This is slightly tricky. Still needs tests.

How did you test this code?

I haven't yet.

@yakkomajuri yakkomajuri changed the base branch from master to events-buffer April 15, 2022 15:00
@yakkomajuri
Copy link
Copy Markdown
Contributor Author

Originally the idea was to use eachMessage for the buffer consumer but that was not flexible enough.

Given I ended up using eachBatch, I may be able to batch both consumers into a single topic-aware consumer?

@yakkomajuri
Copy link
Copy Markdown
Contributor Author

Closed in favor of #9432

@yakkomajuri
Copy link
Copy Markdown
Contributor Author

Big shoutout for kafkajs seamless support for pausing consumption from a particular partition in a topic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant