Skip to content

Standardise log levels#1150

Closed
andrewlock wants to merge 3 commits into
masterfrom
andrew/log-levels
Closed

Standardise log levels#1150
andrewlock wants to merge 3 commits into
masterfrom
andrew/log-levels

Conversation

@andrewlock

Copy link
Copy Markdown
Member

Initial cleanup of log levels before implementing remainder of the logging RFC

  • Use only the allowed levels: Debug, Warning, Error
  • Update default/minimum log level accordingly
  • Update some log levels based on suggestions in RFC.
  • Replace string.Format() style log templates ("{0},{1}") with named arguments ("{Name},{Value}"). Doesn't make much difference atm, as we're only writing to file, but is better practice, and will be better if we do structured logging down the line instead.

In a follow up PR I'll encapsulate the ILogger to restrict to using only the allowed levels, and add rate limiting, as defined in the RFC

@DataDog/apm-dotnet

@andrewlock
andrewlock requested a review from a team as a code owner January 20, 2021 10:12
@andrewlock
andrewlock force-pushed the andrew/log-levels branch 2 times, most recently from 0686afd to 4dbb31d Compare January 22, 2021 09:24

@tonyredondo tonyredondo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tonyredondo
tonyredondo requested a review from a team January 22, 2021 17:35
Comment thread src/Datadog.Trace/Tracer.cs Outdated
if (count >= _maxTracesPerInterval)
{
Log.Debug("Dropping trace id {0} with count of {1} for last {2}ms.", span.TraceId, count, _intervalMilliseconds);
Log.Warning("Dropping trace id {TraceId} with count of {Count} for last {Interval}ms.", span.TraceId, count, _intervalMilliseconds);

@colin-higgins colin-higgins Jan 22, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be very noisy if it's a warning level. Many applications exceed the maximum.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*Shakes fist at RFC*

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the rate limiting API, (in a separate PR atm) this shouldn't be a problem

Comment thread src/Datadog.Trace/Agent/TransportStrategy.cs Outdated

@lucaspimentel lucaspimentel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replacing {0},{1} with {name1},{name2} is great! This always bothered me and I don't remember why we didn't do this before.

If you split that change into a separate PR, I would approve it on the spot. However, some of the other changes in this PR need further discussion. I think we need to keep the Information level around for log messages we want to emit by default but are not Warning or Error. It may be time to update that RFC 😅

}

Log.Debug("ISpanBuilder.AddReference is not implemented for other references than ChildOf by Datadog.Trace");
Log.Warning("ISpanBuilder.AddReference is not implemented for other references than ChildOf by Datadog.Trace");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may sound weird, but in the OpenTracing library we intentionally don't want to log warnings when unimplemented features are used (other examples). We originally logged them, but we received user complaints about spamming their logs. Many times users combine this library with another 3rd party library they may use features like ISpan.Log() which the user cannot disable.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then again, if we're adding rate limiting, spamming the logs should not be an issue anymore.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree, it was just that it was explicitly called out in the RFC (as with many of the other issues here) 🙂

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left it as debug for now - doesn't seem worth updating to warning if people have actively complained about it!

if (!success)
{
Log.Debug("Trace buffer is full. Dropping a trace from the buffer.");
Log.Warning("Trace buffer is full. Dropping a trace from the buffer.");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably check with the other tracer teams whether we should log this message as debug or warning. It's expected that we never send more than 1,000 tracer per second to the Agent. (There's an undocumented setting to change this number.)

Comment thread src/Datadog.Trace/Sampling/DefaultSamplingRule.cs Outdated
if (count >= _maxTracesPerInterval)
{
Log.Debug("Dropping trace id {0} with count of {1} for last {2}ms.", span.TraceId, count, _intervalMilliseconds);
Log.Warning("Dropping trace id {TraceId} with count of {Count} for last {Interval}ms.", span.TraceId, count, _intervalMilliseconds);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to other comments I left, we should check with the larger APM Integrations team about the severity of this message.

Comment thread src/Datadog.Trace/Tracer.cs Outdated
Comment thread src/Datadog.Trace/TracingProcessManager.cs Outdated
@andrewlock

Copy link
Copy Markdown
Member Author

Replacing {0},{1} with {name1},{name2} is great! This always bothered me and I don't remember why we didn't do this before.

If you split that change into a separate PR, I would approve it on the spot.

@lucaspimentel I created a separate PR here: #1163

The RFC only uses Debug, Warn, and Error. We add Information, for  emitting non-error logs (such as configuration status logs)
@andrewlock

Copy link
Copy Markdown
Member Author

Closing as work has been and partially reverted, and otherwise incorporated into #1163 and #1153

@andrewlock andrewlock closed this Jan 29, 2021
@andrewlock
andrewlock deleted the andrew/log-levels branch January 29, 2021 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants