Skip to content

Emit log message for incompatible Lograge setup #3839

Merged
marcotc merged 1 commit into
masterfrom
lograge-rails
Aug 13, 2024
Merged

Emit log message for incompatible Lograge setup #3839
marcotc merged 1 commit into
masterfrom
lograge-rails

Conversation

@marcotc

@marcotc marcotc commented Aug 9, 2024

Copy link
Copy Markdown
Member

Closes #2012

This PR emits an error-level log message when the lograge gem is used with Rails tagged logging, which is configured by default in Rails since Rails 5.

We can't practically fix the issue here in dd-trace-rb, since the issue is a log formatting conflict between lograge and Rails.
The reason this affects APM Tracing is because it breaks trace-log correlation.

The only alternatives are to either: disable Rails tagged logging; or not use lograge.

The investigation and suggest alternatives are well documented here: #2012 (comment)

@github-actions github-actions Bot added integrations Involves tracing integrations tracing labels Aug 9, 2024
@pr-commenter

pr-commenter Bot commented Aug 9, 2024

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2024-08-12 23:14:11

Comparing candidate commit 482b48a in PR branch lograge-rails with baseline commit 6b50227 in branch master.

Found 2 performance improvements and 0 performance regressions! Performance is the same for 21 metrics, 2 unstable metrics.

scenario:profiler - Allocations ()

  • 🟩 throughput [+165850.392op/s; +173043.914op/s] or [+4.230%; +4.414%]

scenario:profiler - sample timeline=false

  • 🟩 throughput [+0.257op/s; +0.396op/s] or [+3.886%; +5.980%]

@marcotc
marcotc force-pushed the lograge-rails branch 2 times, most recently from 83c0965 to d7e71dc Compare August 12, 2024 22:04
@marcotc marcotc changed the title Lograge warning Emit log message for incompatible Lograge setup Aug 12, 2024
@marcotc
marcotc marked this pull request as ready for review August 12, 2024 22:34
@marcotc
marcotc requested a review from a team as a code owner August 12, 2024 22:34
@codecov-commenter

codecov-commenter commented Aug 12, 2024

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (6b50227) to head (482b48a).
⚠️ Report is 5232 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3839      +/-   ##
==========================================
- Coverage   97.84%   97.84%   -0.01%     
==========================================
  Files        1264     1264              
  Lines       75682    75696      +14     
  Branches     3720     3723       +3     
==========================================
+ Hits        74053    74061       +8     
- Misses       1629     1635       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@armstrjare

Copy link
Copy Markdown

We've started getting this error message since this update.

However, we already account for this with customisations to the log formatter to output in JSON and include Datadog tags:
Part of our implementation for this is made public here: https://github.com/armstrjare/rails-json-tagged-logging

Can we add a way to disable this warning to stop polluting our startup logs?

@marcotc @TonyCTHsu

@alexevanczuk

Copy link
Copy Markdown
Contributor

Hey @marcotc just wanted to +1 for a way to disable this warning message.

I'm also confused about the exact behavior that's broken. We set up some log remapping that ensures the datadog trace IDs are properly read as trace IDs and we still have log correlation working. Is there anything else that might be missing with lograge that I'm not seeing?

Thank you!

@marcotc

marcotc commented Jan 8, 2025

Copy link
Copy Markdown
Member Author

Hey @armstrjare and @alexevanczuk, thank you for reporting this issue. It makes total sense.

I opened #4268 so we can track the work of adding an option to disable this log message.

@alexevanczuk

Copy link
Copy Markdown
Contributor

Thanks @marcotc !! Also let me know if there's anything that lograge makes not possible besides log correlation (which we 'roll ourselves' by putting the trace ID into the lograge logged message)!

Thanks again!

@tcannonfodder

Copy link
Copy Markdown

We've started getting this error message since this update.

However, we already account for this with customisations to the log formatter to output in JSON and include Datadog tags: Part of our implementation for this is made public here: https://github.com/armstrjare/rails-json-tagged-logging

Can we add a way to disable this warning to stop polluting our startup logs?

@marcotc @TonyCTHsu

Warning: If upgrading from Rails 7.1 => 7.2 (even if you have a working JSON tagged logging setup using lograge + rails-json-tagged-logging), your logging setup will break, which seems to also cause a memory leak. I wasn't able to debug exactly what broke; I instead switched everything over to rails_semantic_logger

@armstrjare

Copy link
Copy Markdown

Warning: If upgrading from Rails 7.1 => 7.2 (even if you have a working JSON tagged logging setup using lograge + rails-json-tagged-logging), your logging setup will break, which seems to also cause a memory leak. I wasn't able to debug exactly what broke; I instead switched everything over to rails_semantic_logger

@tcannonfodder I’m still using this in production without issue with a decent request volume - and also was on 7.2 when it was released. Happy to help if you throw an issue over at my repo, but sounds like you were happy to just move to semantic logger anyway 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integrations Involves tracing integrations tracing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lograge in JSON mode in Rails causes double addition of log correlation information and breaks log parsing

6 participants