-
-
Notifications
You must be signed in to change notification settings - Fork 827
Description
#7519 has a blocker: a socket only has 2 channels (one per direction), so we can't easily map the stdout/stderr streams onto 2 channels in same direction.
borg.repository code uses the standard python logging calls and we used to set up the logger for remote repositories so it emits json lines on stderr.
btw, there always has been one slight issue with using stderr log output of remote repositories: the ssh client invoked as a subprocess may also emit error messages on stderr, so it sometimes was a bit confusing what came from the remote side and what came from the local ssh process.
we could set up a different handler: https://docs.python.org/3.9/library/logging.handlers.html :
- https://docs.python.org/3.9/library/logging.html#logging.Handler maybe subclass this
- https://docs.python.org/3.9/library/logging.handlers.html#logging.handlers.BufferingHandler maybe subclass this
- https://docs.python.org/3.9/library/logging.handlers.html#queuehandler and https://docs.python.org/3.9/library/logging.handlers.html#queuelistener (uses an extra thread)
In the end, the log data would have to go over stdout, multiplexed with the rpc results also using stdout.
An RPC call comes in on stdin as a msgpacked dict with MSGID and method/args, the call result gets sent on stdout as a msgpacked dict with same MSGID and RESULT data (or same MSGID and "exception_class" ... in case of an exception).
On server side, log data could be also sent on stdout as a msgpacked dict:
- no MSGID (we do not have a msgid when doing a logging call, so we can't have this in the dict)
- logger name
- level, message, ...
On client side, log data is received, unpacked and a respective call to logging is made (logging.getLogger(name) / logger.log(level, msg, *args, **kwargs)`).
This also means: going away from json log serialisation in RepositoryServer and replace this with msgpacked log data.
On the client side, the log data is fed into the logger and then output in the desired way (could be normal log lines or json log lines), depending on how the clientside logger is configured.