Skip to content

stop using stderr for log output of remote repositories #7604

@ThomasWaldmann

Description

@ThomasWaldmann

#7519 has a blocker: a socket only has 2 channels (one per direction), so we can't easily map the stdout/stderr streams onto 2 channels in same direction.

borg.repository code uses the standard python logging calls and we used to set up the logger for remote repositories so it emits json lines on stderr.

btw, there always has been one slight issue with using stderr log output of remote repositories: the ssh client invoked as a subprocess may also emit error messages on stderr, so it sometimes was a bit confusing what came from the remote side and what came from the local ssh process.

we could set up a different handler: https://docs.python.org/3.9/library/logging.handlers.html :

In the end, the log data would have to go over stdout, multiplexed with the rpc results also using stdout.

An RPC call comes in on stdin as a msgpacked dict with MSGID and method/args, the call result gets sent on stdout as a msgpacked dict with same MSGID and RESULT data (or same MSGID and "exception_class" ... in case of an exception).

On server side, log data could be also sent on stdout as a msgpacked dict:

  • no MSGID (we do not have a msgid when doing a logging call, so we can't have this in the dict)
  • logger name
  • level, message, ...

On client side, log data is received, unpacked and a respective call to logging is made (logging.getLogger(name) / logger.log(level, msg, *args, **kwargs)`).

This also means: going away from json log serialisation in RepositoryServer and replace this with msgpacked log data.
On the client side, the log data is fed into the logger and then output in the desired way (could be normal log lines or json log lines), depending on how the clientside logger is configured.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions