Merged
Conversation
This reverts commit 9e0fa57, reversing changes made to 604c258. Signed-off-by: Tamás Kosztyu <[email protected]>
sodomelle
force-pushed
the
revert-s3-destination-changes
branch
from
969db14 to
3d224d8
Compare
sodomelle
added a commit
to sodomelle/axosyslog
that referenced
this pull request
Nov 11, 2025
Signed-off-by: Tamás Kosztyu <[email protected]>
Contributor
|
We've reverted the change because storing chunks in memory is not acceptable for our use cases. S3 objects, in general, are not appendable; the overall object size is huge, so we expect relatively big chunks as well. |
MrAnno
reviewed
Nov 11, 2025
This commit applies parts of 1c073f71a7b942d835fafe1b82d0267589c418ef, commited by Bálint Horváth into to the syslog-ng codebase. Signed-off-by: Tamás Kosztyu <[email protected]>
Creating an empty chunk and waiting for messages to start filling it later caused a bug where empty chunks were created, and instantly uploaded. Boto3 has a bug, where uploading empty files causes issues with uploading. Since our bookkeeping showed that there are chunks missing from the uploaded object, the object wasn't finished on the destination. Now we start a new chunk, only if there is incoming message, so no empty chunks can be uploaded. Signed-off-by: Tamás Kosztyu <[email protected]>
Signed-off-by: Tamás Kosztyu <[email protected]>
sodomelle
force-pushed
the
revert-s3-destination-changes
branch
from
16a41fc to
ef71251
Compare
MrAnno
approved these changes
Nov 11, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR reverts the S3 related changes taken from syslog-ng.
Fixing a bug where empty chunks were uploaded, which caused problems with object creation on the destination side.
Implements the
object_key_suffixoption introduced by Bálint Horváth into the syslog-ng.