Fix issues with tailing rotated jsonlog file#24514
Merged
aaronlehmann merged 1 commit intomoby:masterfrom Aug 26, 2016
Merged
Conversation
This was referenced Jul 11, 2016
cpuguy83
force-pushed
the
fix_log_follow_rotate
branch
2 times, most recently
from
97e4ce2 to
8d6f2e3
Compare
|
+1, fixed the issue for me on ARM64. |
Contributor
|
LGTM |
|
Can this get merged? |
Member
|
ping @tonistiigi @aaronlehmann PTAL |
daemon/logger/jsonfilelog/read.go
Outdated
Fixes a race where the log reader would get events for both an actual rotation as we from fsnotify (`fsnotify.Rename`). This issue becomes extremely apparent when rotations are fast, for example: ``` $ docker run -d --name test --log-opt max-size=1 --log-opt max-file=2 busybox sh -c 'while true; do echo hello; usleep 100000; done' ``` With this change the log reader for jsonlogs can handle rotations that happen as above. Instead of listening for both fs events AND rotation events simultaneously, potentially meaning we see 2 rotations for only a single rotation due to channel buffering, only listen for fs events (like `Rename`) and then wait to be notified about rotation by the logger. This makes sure that we don't see 2 rotations for 1, and that we don't start trying to read until the logger is actually ready for us to. Signed-off-by: Brian Goff <[email protected]>
cpuguy83
force-pushed
the
fix_log_follow_rotate
branch
from
8d6f2e3 to
84e60a7
Compare
Member
Author
|
ping @aaronlehmann |
|
LGTM |
|
I am having same problem.. I am user master tag. is it available in master tag? |
Member
Author
|
@suchisubhra Yes, it is in master. Thanks! |
liusdu
pushed a commit
to liusdu/moby
that referenced
this pull request
Oct 30, 2017
Fixes a race where the log reader would get events for both an actual rotation as we from fsnotify (`fsnotify.Rename`). This issue becomes extremely apparent when rotations are fast, for example: ``` $ docker run -d --name test --log-opt max-size=1 --log-opt max-file=2 busybox sh -c 'while true; do echo hello; usleep 100000; done' ``` With this change the log reader for jsonlogs can handle rotations that happen as above. Instead of listening for both fs events AND rotation events simultaneously, potentially meaning we see 2 rotations for only a single rotation due to channel buffering, only listen for fs events (like `Rename`) and then wait to be notified about rotation by the logger. This makes sure that we don't see 2 rotations for 1, and that we don't start trying to read until the logger is actually ready for us to. backport from moby#24514 conflict: daemon/logger/jsonfilelog/read.go Signed-off-by: Brian Goff <[email protected]> Signed-off-by: Yuanhong Peng <[email protected]>
liusdu
pushed a commit
to liusdu/moby
that referenced
this pull request
Oct 30, 2017
Fix issues with tailing rotated jsonlog file Fixes a race where the log reader would get events for both an actual rotation as we from fsnotify (`fsnotify.Rename`). This issue becomes extremely apparent when rotations are fast, for example: ``` $ docker run -d --name test --log-opt max-size=1 --log-opt max-file=2 busybox sh -c 'while true; do echo hello; usleep 100000; done' ``` With this change the log reader for jsonlogs can handle rotations that happen as above. Instead of listening for both fs events AND rotation events simultaneously, potentially meaning we see 2 rotations for only a single rotation due to channel buffering, only listen for fs events (like `Rename`) and then wait to be notified about rotation by the logger. This makes sure that we don't see 2 rotations for 1, and that we don't start trying to read until the logger is actually ready for us to. backport from moby#24514 conflict: daemon/logger/jsonfilelog/read.go Signed-off-by: Brian Goff <[email protected]> Signed-off-by: Yuanhong Peng <[email protected]> See merge request docker/docker!592
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes a race where the log reader would get events for both an actual
rotation and from fsnotify (
fsnotify.Rename).This issue becomes extremely apparent when rotations are fast, for
example:
With this change the log reader for jsonlogs can handle rotations that
happen as above.
Instead of listening for both fs events AND rotation events
simultaneously, potentially meaning we see 2 rotations for only a single
rotation due to channel buffering, only listen for fs events (like
Rename) and then wait to be notified about rotation by the logger.This makes sure that we don't see 2 rotations for 1, and that we don't
start trying to read until the logger is actually ready for us to.
Fixes #21220
Fixes #23913