Repository navigation
Description
Is there an existing issue already for this bug?
- I have searched for an existing issue, and could not find anything. I believe this is a new bug.
I have read the troubleshooting guide
- I have read the troubleshooting guide and I think this is a new bug.
I am running a supported version of CloudNativePG
- I have read the troubleshooting guide and I think this is a new bug.
Contact Details
Version
trunk (main)
What version of Kubernetes are you using?
1.35
What is your Kubernetes environment?
Self-managed: kind (evaluation)
How did you install the operator?
YAML manifest
What happened?
Summary
In prettyCmd.group(), a new time.Timer is created and a defer timer.Stop() is registered on every iteration of an unbounded for loop. Because defer runs only when the function returns — not at the end of each iteration — the timers and their deferred Stop() calls accumulate for the entire lifetime of the log stream, causing unbounded memory growth.
This was found by code inspection on the main branch; it is environment-independent.
Affected code
internal/cmd/plugin/logs/pretty/pretty.go, function group() (line 145); the defect is the for loop at lines 163-165:
logLoop:
for {
timer := time.NewTimer(1 * time.Second)
defer timer.Stop() // accumulates; only runs when group() returns
select {
case <-ctx.Done():
break logLoop
case <-timer.C:
pushLogGroup()
case logRecord, ok := <-logChannel:
...
}
}Why it happens
defer executes when the enclosing function returns, not at the end of each loop iteration. The loop runs once per incoming log record, so each record creates a fresh time.Timer and pushes another deferred Stop() onto the stack. None of them are released until group() returns, i.e. until the whole log stream ends. The deferred closures also retain a reference to each timer, preventing garbage collection.
Impact
For a long-running stream such as kubectl cnpg logs cluster <name> -f on a busy cluster (or piping a large historical log through pretty), memory and the deferred-call stack grow without bound, proportional to the number of log lines processed.
Secondary effect: because each iteration installs a brand-new 1-second timer, the inactivity-based flush is effectively reset on every record.
Steps to reproduce
- Stream a high volume of logs through the command, e.g.
kubectl cnpg logs cluster <name> -f | kubectl cnpg logs pretty, on a busy cluster. - Observe the process RSS grow steadily and unbounded with the number of log lines, instead of staying flat.
Isolated reproduction (mirroring the loop structure): feeding 1000 records produces 1001 time.NewTimer allocations whose Stop() calls do not run until the loop exits.
Expected behaviour
Memory usage should stay flat regardless of how many log lines are streamed; at most one timer should exist at a time.
Suggested fix
Create the timer once before the loop and Reset it inside, or use a single time.Ticker created before the loop with defer ticker.Stop() registered once outside the loop.
Additional context
The buggy line has existed unchanged since the logs pretty command was introduced (#5779). No open or closed issue/PR currently covers it.
Are you willing to contribute?
Yes — I'd like to be assigned to this. My plan: move the timer creation out of the loop (single time.Timer created before for, drained and Reset on each iteration, with defer timer.Stop() registered once), and confirm the inactivity-flush behaviour is preserved.
Cluster resource
N/A — found by source-code inspection, not from a running cluster.Relevant log output
N/A — code-level defect; no runtime logs apply.Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
dosubot commentedon Jun 13, 2026
Good catch! I can confirm the issue exists in the current code. The
group()function ininternal/cmd/plugin/logs/pretty/pretty.godoes indeed create a newtime.Timerand register adefer timer.Stop()on every loop iteration [1]. Sincedeferonly fires when the enclosing function returns, all those timers and deferred calls pile up for the lifetime of the stream.There are no existing PRs or issues addressing this — the most recent change to the file was PR #10008 (March 2026), which only added a linter suppression [2].
Your proposed fix (single
time.Timerbefore the loop, drained+Resetper iteration, onedeferoutside) is the idiomatic approach and should preserve the 1-second inactivity flush semantics while keeping memory flat.To reply, just mention @dosu.
Docs are dead. Just use Dosu.
Anand-240 commentedon Jun 13, 2026
I'd like to work on this.
Anand-240 commentedon Jun 20, 2026
Hi @gbartolini, just following up on this — I'd love to work on the fix if it's available. My plan is to replace the per-iteration time.NewTimer with a single time.Ticker created before the loop, with defer ticker.Stop() registered once. Let me know if I can proceed!
fix(plugin): avoid per-iteration timer allocation in logs pretty
fix(plugin): fix memory leak when following logs with kubectl cnpg lo…
fix(plugin): fix memory leak when following logs with kubectl cnpg lo…
fix(plugin): fix memory leak when following logs with kubectl cnpg lo…
fix(plugin): fix memory leak when following logs with kubectl cnpg lo…