Skip to content

journal: add concept of "journal namespaces"#14178

Merged
poettering merged 49 commits into
systemd:masterfrom
poettering:journal-namespace
Feb 1, 2020
Merged

journal: add concept of "journal namespaces"#14178
poettering merged 49 commits into
systemd:masterfrom
poettering:journal-namespace

Conversation

@poettering

@poettering poettering commented Nov 28, 2019

Copy link
Copy Markdown
Member

This allows multi-instantiation of journal daemons, for logical separation and performance improvement.

Mostly an implementaiton of #12123.

@lgtm-com

lgtm-com Bot commented Nov 28, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 74bc8c5 into d2a5659 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@lgtm-com

lgtm-com Bot commented Nov 28, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 61f9cd5 into d2a5659 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@lgtm-com

lgtm-com Bot commented Nov 28, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 16fa91d into 870a2d8 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@lgtm-com

lgtm-com Bot commented Nov 28, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 20fc0f8 into 88eb051 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@lgtm-com

lgtm-com Bot commented Nov 28, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 70cec3e into 88eb051 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

Comment thread man/systemd-journald.service.xml Outdated
Comment thread man/systemd.exec.xml
@poettering

Copy link
Copy Markdown
Member Author

force pushed a new version with issues @boucman noticed fixed! thanks!

@lgtm-com

lgtm-com Bot commented Dec 2, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 62adc71 into cb51560 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@lgtm-com

lgtm-com Bot commented Dec 4, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging ea1d14d into 26601a2 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@lgtm-com

lgtm-com Bot commented Dec 4, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 9d559b7 into 41ab3b7 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@anitazha anitazha left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really excited for this feature! Looks good, though I only skimmed the varlink stuff. Some minor comments.

(Also I would really appreciate if you would split the refactoring stuff into a different PR so it would be easier to focus on the feature changes vs the style/refactors...)

Comment thread src/journal/journald-server.c
Comment thread src/journal/journald-context.c Outdated
Comment thread src/journal/journald-server.c
@poettering

Copy link
Copy Markdown
Member Author

force pushed new version with the typo @anitazha pointed out fixed.

Comment thread man/journalctl.xml
</varlistentry>

<varlistentry>
<term><option>--namespace=<replaceable>NAMESPACE</replaceable></option></term>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I do something like: +webserver,db,msgbus and have those three namespaces added to the default shown?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, currently you specify one namespace and then optionally get the default one added in, that's all. Extending this eventually to a list of namespaces is pretty easy though, but let's do that later.

Comment thread man/journald.conf.xml
<refnamediv>
<refname>journald.conf</refname>
<refname>journald.conf.d</refname>
<refname>[email protected]</refname>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is [email protected] vs journald@<NAMESPACE>.conf? Is that the base template file that you build into a namespace configuration?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that means nothing. It's just that the man page name is generated from what is specified here, and including <NAMESPACE> in the man page name makes it hard to type in the shell... Hence I figured we just write in in the template syntax systemd uses for units, even though it's a bit sloppy, since we don't do templating here...

Comment thread man/systemd-journald.service.xml Outdated
@lgtm-com

lgtm-com Bot commented Dec 9, 2019

Copy link
Copy Markdown

This pull request introduces 1 alert when merging 568ae03 into fb4b046 - view on LGTM.com

new alerts:

  • 1 for FIXME comment

Comment thread man/systemd-journald.service.xml Outdated
Comment thread man/systemd-journald.service.xml Outdated
Comment thread man/systemd.exec.xml
If we have exit on idle, then operations such as "journalctl
--namespace=foo --rotate" should work even if the journal daemon is
currently not running.

(Note that we don't do activation by varlink for the main instance of
journald, I am not sure the deadlocks it might introduce are worth it)
We already apply them to the directory in /var. Let's do the same in
/run too. That's because due to the log namespace logic we nowadays can
gain additional subdirs there during regular operation.
It doesn't get us much, but makes the differences between the templated
and non-templated versions a bit smaller.
@poettering

Copy link
Copy Markdown
Member Author

Force pushed a new version now. Addressed everything pointed out, except for the stuff I commented on, see above. No other changes.

@lgtm-com

lgtm-com Bot commented Jan 31, 2020

Copy link
Copy Markdown

This pull request introduces 1 alert when merging ad23785 into 9a1862b - view on LGTM.com

new alerts:

  • 1 for FIXME comment

@poettering

Copy link
Copy Markdown
Member Author

Given that I only fixed minor stuff in the last round, and the positive reviews from multiple reviewers, and the points left to discuss can still be done after merge, and the CI is all green, taking liberty to merge this! Thanks everyone for the reviews!

@poettering
poettering merged commit 5ee69e1 into systemd:master Feb 1, 2020
@evverx

evverx commented Feb 1, 2020

Copy link
Copy Markdown
Contributor

Trying to run TEST-44-* under ASan/UBsan I noticed that it's flaky. As it turns out, "hello world" doesn't always end up anywhere and judging by the logs I managed to get from the container sometimes logs are just lost:

systemd-testsuite systemd[1]: [email protected]: Incoming traffic
systemd-testsuite systemd[1]: [email protected]: Suppressing connection request since unit stop is scheduled.
systemd-testsuite systemd[1]: [email protected]: Incoming traffic
systemd-testsuite systemd[1]: [email protected]: Suppressing connection request since unit stop is scheduled.
systemd-testsuite systemd[1]: [email protected]: Succeeded.

I suspect it has something to do with short-lived processes because something like

systemctl start systemd-journald@foobar
systemd-run -p LogNamespace=foobar echo 'hello world'

or

systemd-run -p LogNamespace=foobar sh -c 'echo hello world; sleep 10'

seems to always work.

I'm also wondering why systemd-journald@* are never restarted? Currently if they vanish into thin air (for whatever reason) services using log namespaces start losing logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

7 participants