Skip to content

Envelope encryption assumes offsets are consecutive.  #841

@k-wall

Description

@k-wall

Describe the bug

EnvelopeEncryptionFilter#recordsBuilder is careful to pass the baseOffset from the incoming records when creating the Builder.
This means we are guaranteed that the first record will have the same offset as was sent by the peer. If the partition has more than one record, subsequent records will be given baseOffset+1 and so forth. This works okay - unless the original offsets are not consecutive (as would be the case with a compacted topic). As things stand I believe the fetch path could have the affect of renumbering the records. This'll cause chaos as the offset accounting will be off.

I think we ought to be using appendWithOffset so that the offsets are preserved.

We ought to have test cases to ensure offset fidelity.

I think Sample might exhibit the same issue.

To Reproduce

I haven't verified this issue. Inference from looking at the code.

Steps to reproduce the behavior:

  1. Configure kroxylicious like this ...
  2. Run command '....'
  3. See error

Expected behavior
A clear and concise description of what you expected to happen.

YAML files and logs

Attach or copy paste your kroxylicious configuration YAML (with sensitive details
like broker address redacted).
Attach or copy and paste relevant logs.

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions