Skip to content

improve handling of trailing newline in journal logging #3636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
evverx merged 4 commits into from
Jul 19, 2016
Merged

improve handling of trailing newline in journal logging #3636

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c24f1f9
sd-journal: when formatting log messages, implicitly strip trailing w…
poettering Jul 1, 2016
8980058
journalctl: make sure that journalctl's --all switch also has an effe…
poettering Jul 1, 2016
4c5db93
man: document that sd_journal_print() strips trailing whitespace
poettering Jul 2, 2016
0d23bc5
sd-journal: suppress empty lines
poettering Jul 19, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 36 additions & 51 deletions man/sd_journal_print.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,27 +93,21 @@
<refsect1>
<title>Description</title>

<para><function>sd_journal_print()</function> may be used to
submit simple, plain text log entries to the system journal. The
first argument is a priority value. This is followed by a format
string and its parameters, similar to
<citerefentry project='man-pages'><refentrytitle>printf</refentrytitle><manvolnum>3</manvolnum></citerefentry>
or
<para><function>sd_journal_print()</function> may be used to submit simple, plain text log entries to the system
journal. The first argument is a priority value. This is followed by a format string and its parameters, similar to
<citerefentry project='man-pages'><refentrytitle>printf</refentrytitle><manvolnum>3</manvolnum></citerefentry> or
<citerefentry project='man-pages'><refentrytitle>syslog</refentrytitle><manvolnum>3</manvolnum></citerefentry>.
The priority value is one of
<constant>LOG_EMERG</constant>,
<constant>LOG_ALERT</constant>,
<constant>LOG_CRIT</constant>,
<constant>LOG_ERR</constant>,
<constant>LOG_WARNING</constant>,
<constant>LOG_NOTICE</constant>,
<constant>LOG_INFO</constant>,
<constant>LOG_DEBUG</constant>, as defined in
<filename>syslog.h</filename>, see
<citerefentry project='man-pages'><refentrytitle>syslog</refentrytitle><manvolnum>3</manvolnum></citerefentry>
for details. It is recommended to use this call to submit log
messages in the application locale or system locale and in UTF-8
format, but no such restrictions are enforced.</para>
The priority value is one of <constant>LOG_EMERG</constant>, <constant>LOG_ALERT</constant>,
<constant>LOG_CRIT</constant>, <constant>LOG_ERR</constant>, <constant>LOG_WARNING</constant>,
<constant>LOG_NOTICE</constant>, <constant>LOG_INFO</constant>, <constant>LOG_DEBUG</constant>, as defined in
<filename>syslog.h</filename>, see <citerefentry
project='man-pages'><refentrytitle>syslog</refentrytitle><manvolnum>3</manvolnum></citerefentry> for details. It is
recommended to use this call to submit log messages in the application locale or system locale and in UTF-8 format,
but no such restrictions are enforced. Note that log messages written using this function are generally not
expected to end in a new-line character. However, as all trailing whitespace (including spaces, new-lines,
tabulators and carriage returns) are automatically stripped from the logged string, it is acceptable to specify one
(or more). Empty lines (after trailing whitespace removal) are suppressed. On non-empty lines, leading whitespace
(as well as inner whitespace) is left unmodified. </para>

<para><function>sd_journal_printv()</function> is similar to
<function>sd_journal_print()</function> but takes a variable
Expand All @@ -123,35 +117,26 @@
for more information) instead of the format string. It is
otherwise equivalent in behavior.</para>

<para><function>sd_journal_send()</function> may be used to submit
structured log entries to the system journal. It takes a series of
format strings, each immediately followed by their associated
parameters, terminated by <constant>NULL</constant>. The strings
passed should be of the format <literal>VARIABLE=value</literal>.
The variable name must be in uppercase and consist only of
characters, numbers and underscores, and may not begin with an
underscore. (All assignments that do not follow this syntax will
be ignored.) The value can be of any size and format. It is highly
recommended to submit text strings formatted in the UTF-8
character encoding only, and submit binary fields only when
formatting in UTF-8 strings is not sensible. A number of
well-known fields are defined, see
<citerefentry><refentrytitle>systemd.journal-fields</refentrytitle><manvolnum>7</manvolnum></citerefentry>
for details, but additional application defined fields may be
used. A variable may be assigned more than one value per
entry.</para>

<para><function>sd_journal_sendv()</function> is similar to
<function>sd_journal_send()</function> but takes an array of
<varname>struct iovec</varname> (as defined in
<filename>uio.h</filename>, see
<citerefentry project='man-pages'><refentrytitle>readv</refentrytitle><manvolnum>3</manvolnum></citerefentry>
for details) instead of the format string. Each structure should
reference one field of the entry to submit. The second argument
specifies the number of structures in the array.
<function>sd_journal_sendv()</function> is particularly useful to
submit binary objects to the journal where that is
necessary.</para>
<para><function>sd_journal_send()</function> may be used to submit structured log entries to the system journal. It
takes a series of format strings, each immediately followed by their associated parameters, terminated by
<constant>NULL</constant>. The strings passed should be of the format <literal>VARIABLE=value</literal>. The
variable name must be in uppercase and consist only of characters, numbers and underscores, and may not begin with
an underscore. (All assignments that do not follow this syntax will be ignored.) The value can be of any size and
format. It is highly recommended to submit text strings formatted in the UTF-8 character encoding only, and submit
binary fields only when formatting in UTF-8 strings is not sensible. A number of well-known fields are defined, see
<citerefentry><refentrytitle>systemd.journal-fields</refentrytitle><manvolnum>7</manvolnum></citerefentry> for
details, but additional application defined fields may be used. A variable may be assigned more than one value per
entry. If this function is used, trailing whitespace is automatically removed from each formatted field.</para>

<para><function>sd_journal_sendv()</function> is similar to <function>sd_journal_send()</function> but takes an
array of <varname>struct iovec</varname> (as defined in <filename>uio.h</filename>, see <citerefentry
project='man-pages'><refentrytitle>readv</refentrytitle><manvolnum>3</manvolnum></citerefentry> for details)
instead of the format string. Each structure should reference one field of the entry to submit. The second argument
specifies the number of structures in the array. <function>sd_journal_sendv()</function> is particularly useful to
submit binary objects to the journal where that is necessary. Note that this function wil not strip trailing
whitespace of the passed fields, but passes the specified data along unmodified. This is different from both
<function>sd_journal_print()</function> and <function>sd_journal_send()</function> described above, which are based
on format strings, and do strip trailing whitespace.</para>

<para><function>sd_journal_perror()</function> is a similar to
<citerefentry project='die-net'><refentrytitle>perror</refentrytitle><manvolnum>3</manvolnum></citerefentry>
Expand All @@ -174,8 +159,8 @@
<programlisting>sd_journal_print(LOG_INFO, "Hello World, this is PID %lu!", (unsigned long) getpid());

sd_journal_send("MESSAGE=Hello World, this is PID %lu!", (unsigned long) getpid(),
"PRIORITY=%i", LOG_INFO,
NULL);</programlisting>
"PRIORITY=%i", LOG_INFO,
NULL);</programlisting>

<para>Note that these calls implicitly add fields for the source
file, function name and code line where invoked. This is
Expand Down
16 changes: 16 additions & 0 deletions src/journal/journal-send.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,13 @@ _public_ int sd_journal_printv(int priority, const char *format, va_list ap) {
memcpy(buffer, "MESSAGE=", 8);
vsnprintf(buffer+8, sizeof(buffer) - 8, format, ap);

/* Strip trailing whitespace, keep prefix whitespace. */
(void) strstrip(buffer);
Copy link
Contributor

@evverx evverx Jul 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we document this?

Copy link
Member Author

@poettering poettering Jul 2, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we should. will do.


/* Suppress empty lines */
if (isempty(buffer+8))
return 0;

zero(iov);
IOVEC_SET_STRING(iov[0], buffer);
IOVEC_SET_STRING(iov[1], p);
Expand Down Expand Up @@ -158,6 +165,8 @@ _printf_(1, 0) static int fill_iovec_sprintf(const char *format, va_list ap, int

VA_FORMAT_ADVANCE(format, ap);

(void) strstrip(buffer); /* strip trailing whitespace, keep prefixing whitespace */
Copy link
Contributor

@evverx evverx Jul 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we shouldn't change the behaviour of the sd_journal_send
I.e. sd_journal_send("MESSAGE=BOO", "VALUE=%s", " ", NULL) should produce:

        "MESSAGE" : "BOO",
        "VALUE" : " ",

instead of

        "MESSAGE" : "BOO",
        "VALUE" : "",

Copy link
Contributor

@evverx evverx Jul 19, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, maybe, this is ok.
LGTM


IOVEC_SET_STRING(iov[i++], buffer);

format = va_arg(ap, char *);
Expand Down Expand Up @@ -471,6 +480,13 @@ _public_ int sd_journal_printv_with_location(int priority, const char *file, con
memcpy(buffer, "MESSAGE=", 8);
vsnprintf(buffer+8, sizeof(buffer) - 8, format, ap);

/* Strip trailing whitespace, keep prefixing whitespace */
(void) strstrip(buffer);

/* Suppress empty lines */
if (isempty(buffer+8))
return 0;

/* func is initialized from __func__ which is not a macro, but
* a static const char[], hence cannot easily be prefixed with
* CODE_FUNC=, hence let's do it manually here. */
Expand Down
4 changes: 2 additions & 2 deletions src/shared/logs-show.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ static int output_verbose(
off = ANSI_NORMAL;
}

if (flags & OUTPUT_SHOW_ALL ||
if ((flags & OUTPUT_SHOW_ALL) ||
(((length < PRINT_CHAR_THRESHOLD) || flags & OUTPUT_FULL_WIDTH)
&& utf8_is_printable(data, length))) {
fprintf(f, " %s%.*s=", on, fieldlen, (const char*)data);
Expand Down Expand Up @@ -607,7 +607,7 @@ void json_escape(
if (!(flags & OUTPUT_SHOW_ALL) && l >= JSON_THRESHOLD)
fputs("null", f);

else if (!utf8_is_printable(p, l)) {
else if (!(flags & OUTPUT_SHOW_ALL) && !utf8_is_printable(p, l)) {
bool not_first = false;

fputs("[ ", f);
Expand Down