Skip to content

Add default log retention settings#8815

Closed
danhermann wants to merge 1 commit intoelastic:masterfrom
danhermann:7482_limit_retained_logs
Closed

Add default log retention settings#8815
danhermann wants to merge 1 commit intoelastic:masterfrom
danhermann:7482_limit_retained_logs

Conversation

@danhermann
Copy link
Copy Markdown
Contributor

@danhermann danhermann commented Dec 7, 2017

The default retention setting of 20 files was arrived at by assuming a conservative 50% compression ratio on rotated log files for a total of 1GB (20 files * 50% compression * 100MB max size). Note that the 1GB limit would apply per appender.

Relates to #7842

@elasticsearch-bot elasticsearch-bot self-assigned this Dec 7, 2017
@jakelandis
Copy link
Copy Markdown
Contributor

I think this should be a 7.x change to avoid changing behavior too much on a minor release. Auto deleting old log files may be seen as non-passive.

Thoughts ?

@jordansissel
Copy link
Copy Markdown
Contributor

Note that the 1GB limit would apply per log file.

Do you mean per appender (slowlog, json, etc), not per log file?

@jakelandis
Copy link
Copy Markdown
Contributor

I would vote to bump to 30

Since we support date or size, which ever comes first... i would assume a (mostly) healthy system would typically hit the date based roll over first. (and I think we will get better then 50% on un-healthy systems)

@danhermann
Copy link
Copy Markdown
Contributor Author

@jordansissel, ah, yes. I meant per appender, not per log file. I'll edit the description.

@danhermann
Copy link
Copy Markdown
Contributor Author

@jakelandis, bumped to 30 files. I agree that we should get >50% compression in most cases.

@jordansissel
Copy link
Copy Markdown
Contributor

I think this should be a 7.x change

I agree with your reasoning. Maybe for 6.x we can offer this as an alternative. Maybe ship a log4j2.properties.rotating or something that people can copy/paste or rename if they wish to use it?

@jordansissel
Copy link
Copy Markdown
Contributor

I don't see any tests added, and maybe that's ok (log4j configs maybe are too complicated to write solid tests for, for this pr). Can someone run a manual test and verify the retention settings work as expected (max 30 files, daily rotation, rotation at 100mb) ?

@jakelandis jakelandis self-requested a review December 8, 2017 14:58
@danhermann
Copy link
Copy Markdown
Contributor Author

Jenkins test this please

@danhermann danhermann force-pushed the 7482_limit_retained_logs branch from 15f3ab3 to 8ee32be Compare December 8, 2017 20:26
@elasticsearch-bot
Copy link
Copy Markdown

dan.hermann merged this into the following branches!

Branch Commits
master 1ea1d19

insukcho pushed a commit to insukcho/logstash that referenced this pull request Feb 1, 2018
@danhermann danhermann deleted the 7482_limit_retained_logs branch April 2, 2019 15:23
@fholzer fholzer mentioned this pull request Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants