Describe the bug
EnvelopeEncryptionFilter#recordsBuilder is careful to pass the baseOffset from the incoming records when creating the Builder.
This means we are guaranteed that the first record will have the same offset as was sent by the peer. If the partition has more than one record, subsequent records will be given baseOffset+1 and so forth. This works okay - unless the original offsets are not consecutive (as would be the case with a compacted topic). As things stand I believe the fetch path could have the affect of renumbering the records. This'll cause chaos as the offset accounting will be off.
I think we ought to be using appendWithOffset so that the offsets are preserved.
We ought to have test cases to ensure offset fidelity.
I think Sample might exhibit the same issue.
To Reproduce
I haven't verified this issue. Inference from looking at the code.
Steps to reproduce the behavior:
- Configure kroxylicious like this ...
- Run command '....'
- See error
Expected behavior
A clear and concise description of what you expected to happen.
YAML files and logs
Attach or copy paste your kroxylicious configuration YAML (with sensitive details
like broker address redacted).
Attach or copy and paste relevant logs.
Additional context
Add any other context about the problem here.
Describe the bug
EnvelopeEncryptionFilter#recordsBuilder is careful to pass the
baseOffsetfrom the incoming records when creating the Builder.This means we are guaranteed that the first record will have the same offset as was sent by the peer. If the partition has more than one record, subsequent records will be given baseOffset+1 and so forth. This works okay - unless the original offsets are not consecutive (as would be the case with a compacted topic). As things stand I believe the fetch path could have the affect of renumbering the records. This'll cause chaos as the offset accounting will be off.
I think we ought to be using
appendWithOffsetso that the offsets are preserved.We ought to have test cases to ensure offset fidelity.
I think Sample might exhibit the same issue.
To Reproduce
I haven't verified this issue. Inference from looking at the code.
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
YAML files and logs
Attach or copy paste your kroxylicious configuration YAML (with sensitive details
like broker address redacted).
Attach or copy and paste relevant logs.
Additional context
Add any other context about the problem here.