Skip to content

add journalctl --sync command and other stuff#1848

Merged
teg merged 6 commits intosystemd:masterfrom
poettering:journal-sync
Nov 11, 2015
Merged

add journalctl --sync command and other stuff#1848
teg merged 6 commits intosystemd:masterfrom
poettering:journal-sync

Conversation

@poettering
Copy link
Member

No description provided.

With this new "--sync" switch we add a synchronous way to sync
everything queued to disk, and return only after that's complete. This
command gives the guarantee that anything queued before has hit the disk
before the command returns.

While we are at it, also improve the man pages and help text for
journalctl a bit.
Of course, ideally we'd just use normal synchronous bus calls, but this
is out of the question as long as we rely on dbus-daemon (which logs to
journald, and thus cannot use to avoid cyclic sync loops). Hence,
instead, reuse the wait logic already implemented for --sync, and use a
signal in one direction, and a mtime watch file for the reply.
teg added a commit that referenced this pull request Nov 11, 2015
add journalctl --sync command and other stuff
@teg teg merged commit a2e6fbf into systemd:master Nov 11, 2015
@evverx
Copy link
Contributor

evverx commented Nov 11, 2015

journalctl --sync; journalctl --sync; journalctl --sync waits forever.
Looks like regular syncs (SyncIntervalSec=) don't update the file /run/systemd/journal/sync

@evverx
Copy link
Contributor

evverx commented Nov 11, 2015

bash-4.3# strace -e ppoll -f journalctl --sync
+++ exited with 0 +++

bash-4.3# strace -e ppoll -f journalctl --sync
+++ exited with 0 +++

bash-4.3# strace -e ppoll -f journalctl --sync
ppoll([{fd=4, events=POLLIN}], 1, NULL, NULL, 8...

@poettering
Copy link
Member Author

@evverx have you restarted your journald? journald only writes the touch files out after the patch was applied...

And yes, we don't touch those files on SyncIntervalSec=, for a reason: otherwise the guarantee we want to give wouldn't hold: we want that everything queued before invocation of "journalctl --sync" is definitely on disk. That requires us that we sync and touch from a low-priority event source, so that all the other log messages (which are dispatched by higher-priorty event sources) are known to be processed already. Now, the sync interval stuff is triggered by a much higher priority event source than the handling of SIGRTMIN+1, which means we better avoid touch the file in that case.

But yeah, we should generate an error if the command is invoked if --machine= is called.

@poettering
Copy link
Member Author

I have prepped a patch locally, that generates an error if --machine= is combined with --flush, --sync or --rotate.

@evverx
Copy link
Contributor

evverx commented Nov 11, 2015

@evverx have you restarted your journald? journald only writes the touch files out after the patch was applied...

I tested in a clean new container: create_new_image; make all; make install DESTDIR=cont-root; ./systemd-nspawd -D cont-root -b systemd.unit=rescue.target

bash-4.3# journalctl --sync
bash-4.3# (sleep 60 && journalctl --sync) &
[1] 35
bash-4.3# strace -e ppoll -f journalctl --sync
ppoll([{fd=4, events=POLLIN}], 1, NULL, NULL, 8
# wait
+++ exited with 0 +++

works fine

@poettering
Copy link
Member Author

OK, so all is good now?

@evverx
Copy link
Contributor

evverx commented Nov 11, 2015

oh, sorry. no.
one of the sequential calls waits forever. i.e.

bash-4.3# strace -e ppoll -f journalctl --sync
+++ exited with 0 +++
bash-4.3# strace -e ppoll -f journalctl --sync
ppoll([{fd=4, events=POLLIN}], 1, NULL, NULL, 8
# wait forever

@evverx
Copy link
Contributor

evverx commented Nov 11, 2015

In this comment, I mean (sleep 60 && journalctl --sync) & unblocks all blocked syncs.

@evverx
Copy link
Contributor

evverx commented Nov 11, 2015

The full output of the strace -f journalctl --sync here.

@poettering
Copy link
Member Author

@evverx i tried hard to reproduce the issue here, but no luck... I ran the --sync stuff in a loop for quite some time, but it worked every time...

The strace you attached shows nothing wrong. For some reason though the tool is never woken up via inotify triggered by a change on /var/run/systemd/synced... Now, this could have two reasons: one could be that there's something wrong with the inotify match we install, and for some reason we don't notice the touch() done by journald. But I don't see how that could happen. The other would be that journald never actually manages to touch the file.

I have the suspicion that the latter is more likely the problem. Could you please use "strace -s500 -p" on the journal pid, and then reproduce the issue? This should show how journald gets the SIGRTMIN+1, and then syncs things to disk and eventually touches the sync file.

@poettering
Copy link
Member Author

@evverx given that this PR is already closed, we should really move this to a proper issue and track it there. I have now created one as #1853. Please attach the requested strace there, and let's end the discussion on this PR. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants