Skip to content

new remote and progress logging, cleanups#7607

Merged
ThomasWaldmann merged 9 commits intoborgbackup:masterfrom
ThomasWaldmann:new-remote-logging
May 29, 2023
Merged

new remote and progress logging, cleanups#7607
ThomasWaldmann merged 9 commits intoborgbackup:masterfrom
ThomasWaldmann:new-remote-logging

Conversation

@ThomasWaldmann
Copy link
Member

No description provided.

@ThomasWaldmann ThomasWaldmann marked this pull request as draft May 26, 2023 02:00
@codecov-commenter
Copy link

codecov-commenter commented May 26, 2023

Codecov Report

Merging #7607 (737d1b5) into master (ca68dd2) will decrease coverage by 0.11%.
The diff coverage is 80.89%.

❗ Current head 737d1b5 differs from pull request most recent head e2ea5cf. Consider uploading reports for the commit e2ea5cf to get more accurate results

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##           master    #7607      +/-   ##
==========================================
- Coverage   83.95%   83.84%   -0.11%     
==========================================
  Files          66       66              
  Lines       11747    11714      -33     
  Branches     2132     2115      -17     
==========================================
- Hits         9862     9822      -40     
- Misses       1321     1333      +12     
+ Partials      564      559       -5     
Impacted Files Coverage Δ
src/borg/logger.py 70.55% <76.66%> (+2.13%) ⬆️
src/borg/remote.py 84.82% <78.57%> (-0.43%) ⬇️
src/borg/archiver/__init__.py 85.53% <100.00%> (+0.04%) ⬆️
src/borg/helpers/__init__.py 100.00% <100.00%> (ø)
src/borg/helpers/progress.py 100.00% <100.00%> (+4.13%) ⬆️

@ThomasWaldmann ThomasWaldmann force-pushed the new-remote-logging branch 2 times, most recently from 652870b to 0538447 Compare May 29, 2023 16:49
Instead, install a handler that sends the LogRecord dicts to a queue.
That queue is then emptied in the borg serve main loop and
the LogRecords are sent msgpacked via stdout to the client,
similar to the RPC results.

On the client side, the LogRecords are recreated from the
received dicts and fed into the clientside logging system.

As we use msgpacked LogRecord dicts, we don't need JSON for
this purpose on the borg serve side any more.
On the client side, the LogRecords will then be either formatted
as normal text or as JSON log output (by the clientside log
formatter).
- simplify progress output (no \r, no terminal size related tweaks)
- emit progress output via the logging system (so it does not use stderr
  of borg serve)
- progress code always logs a json string, the json has all needed
  to either do json log output or plain text log output.
- use formatters to generate plain or json output from that.
- clean up setup_logging
- use a StderrHandler that always uses the **current** sys.stderr
- tweak TestPassphrase to not accidentally trigger just because of seeing 12 in output
for normal borg command invocation:
- logging is set up in Archiver.run
- the atexit handler calls logging.shutdown when process terminates

for tests:
- Archiver.run called by exec_cmd
- no atexit handler executed as process lives on
- borg.logger.teardown (calls shutdown and configured=False) now
  called in exec_cmd
- tears down logging (so no new log output is generated afterwards)
- sends all queued log output
- then returns

also: make stdin_fd / stdout_fd instance variables
@ThomasWaldmann ThomasWaldmann marked this pull request as ready for review May 29, 2023 20:41
@ThomasWaldmann ThomasWaldmann changed the title new logging / cleanups new remote and progress logging, cleanups May 29, 2023
also:
add missing param to RemoteRepositoryTestCase.open method, but ignore it.
@ThomasWaldmann ThomasWaldmann merged commit 87b74f3 into borgbackup:master May 29, 2023
@ThomasWaldmann ThomasWaldmann deleted the new-remote-logging branch May 29, 2023 22:14
snorkelopstesting2-coder pushed a commit to snorkel-marlin-repos/borgbackup_borg_pr_7607_aad6a490-d038-40f9-8280-1f5bec90c0ac that referenced this pull request Oct 22, 2025
Original PR #7607 by ThomasWaldmann
Original: borgbackup/borg#7607
snorkelopsstgtesting1-spec pushed a commit to snorkel-marlin-repos/borgbackup_borg_pr_7607_9c695c36-8605-49a2-aca5-41c4c6629f52 that referenced this pull request Oct 22, 2025
Original PR #7607 by ThomasWaldmann
Original: borgbackup/borg#7607
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants