Skip to content

Fix disappearing metrics-probe() counters#243

Merged
MrAnno merged 11 commits intoaxoflow:mainfrom
MrAnno:fix-metrics-probe-cntr-lifetime
Aug 13, 2024
Merged

Fix disappearing metrics-probe() counters#243
MrAnno merged 11 commits intoaxoflow:mainfrom
MrAnno:fix-metrics-probe-cntr-lifetime

Conversation

@MrAnno
Copy link
Contributor

@MrAnno MrAnno commented Aug 8, 2024

The TLS cache of metrics-probe() has been extended with a global cache in order not to make metrics orphaned whenever an ivykis worker stops (after 10 seconds of inactivity).

The global cache requires a lock, but it is only updated on thread shutdown. It is purged during reload.

Refactor overview:

  • metrics-cache --> dyn-metrics-store
  • metrics-tls-cache --> dyn-metrics-cache

I've also tried to decouple metrics-template from the cache functionality, but it would have had performance implications, so I only added a comment about the coupled functionality.

This is the first step of a refactor round that tries to decouple a few
dynamic-metric-related units from the cache built around it.

The end goal is to fix a lifetime issue that is related to the TLS cache
version of this store.

Signed-off-by: László Várady <[email protected]>
MrAnno added a commit to MrAnno/axosyslog that referenced this pull request Aug 8, 2024
Signed-off-by: László Várady <[email protected]>
@MrAnno MrAnno force-pushed the fix-metrics-probe-cntr-lifetime branch from 78b9ffe to 9e04f46 Compare August 8, 2024 07:54
MrAnno added 3 commits August 8, 2024 11:15
It is no longer called a "cache", so "retrieve" may express the
functionality better.

Signed-off-by: László Várady <[email protected]>
This is a preparation step for fixing a lifetime bug related to TLS
caching. This unit will be extended with a global cache in order not to
make metrics orphaned whenever an ivykis worker stops.

Signed-off-by: László Várady <[email protected]>
MrAnno added a commit to MrAnno/axosyslog that referenced this pull request Aug 8, 2024
Signed-off-by: László Várady <[email protected]>
@MrAnno MrAnno force-pushed the fix-metrics-probe-cntr-lifetime branch from 9e04f46 to 4e1745c Compare August 8, 2024 09:15
@MrAnno
Copy link
Contributor Author

MrAnno commented Aug 8, 2024

@kira-syslogng do stresstest

Okay, I'm stupid :D

MrAnno added 7 commits August 8, 2024 11:56
To denote that this is about templating dynamic counters.

Signed-off-by: László Várady <[email protected]>
It would have been better to decouple the metrics template unit from the
cache, but this would have had performance implications.

Signed-off-by: László Várady <[email protected]>
Signed-off-by: László Várady <[email protected]>
This is required for merging the TLS-cache into a global cache to avoid
making dynamic metrics orphan on thread shutdown.

Signed-off-by: László Várady <[email protected]>
This unit has been extended with a global cache in order not to
make metrics orphaned whenever an ivykis worker stops (after 10 seconds of
inactivity).

The global cache requires a lock, but it is only updated on thread
shutdown.

The global cache is purged during reload.

Signed-off-by: László Várady <[email protected]>
Signed-off-by: László Várady <[email protected]>
@MrAnno MrAnno force-pushed the fix-metrics-probe-cntr-lifetime branch from 4e1745c to 10376b4 Compare August 8, 2024 09:56
Copy link
Member

@bazsi bazsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Great work.

@MrAnno MrAnno merged commit d356f61 into axoflow:main Aug 13, 2024
HofiOne pushed a commit to HofiOne/axosyslog that referenced this pull request Mar 13, 2025
Signed-off-by: László Várady <[email protected]>
Signed-off-by: Hofi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants