Join the Preview!
Log Optimizer is in Preview. To optimize your log volumes and reduce noise in your environment, reach out to your account manager.
Overview
Log Optimizer helps you identify log patterns that generate high volumes of repetitive or noisy data. Datadog analyzes your indexed logs and recommends actions, such as excluding, sampling, or converting logs to metrics, so you can optimize log volumes and focus on the most relevant information for troubleshooting and analysis.
This feature builds on Logging without Limits™ and complements tools such as Exclusion Filters and Logs to Metrics.
How it works
Datadog continuously reviews your indexed logs to find patterns that generate large or repetitive volumes of data. Once per day, Log Optimizer evaluates these patterns against Datadog best practices and identifies logs that may benefit from optimization.
The Log Optimizer then suggests actions (such as excluding debug-level messages, sampling routine logs, or converting static messages to metrics) so you can reduce noise without losing visibility into important events.
The Log Optimizer does not account for existing exclusion filters or logs-to-metrics conversions. Review your settings before applying new actions to avoid duplicates.
What Datadog analyzes
- Indexed logs: The analysis targets logs stored in your Standard and Flex indexes.
- High-volume patterns: Datadog detects patterns that make up a significant share of your total log volume.
- Message consistency and content: Logs with repeated or low-variability messages are evaluated as potential candidates for optimization. For example, if log messages indicate successful operations (such as “process executed successfully”), Log Optimizer may recommend excluding those logs to reduce noise.
- Monitor usage across the platform: Datadog checks the recommended patterns against your list of active monitors to show you where your logs are used.
Recommended actions
Each recommendation includes an explanation and a suggested action.
| Recommendation | Description | Typical example |
|---|
| Exclude | Stop indexing logs that add noise and make it harder to focus on critical signals. | Debug-level messages or verbose system output. |
| Sample | Lower the percentage of repetitive logs to reduce noise without losing visibility. | Logs with very little variability (where fields like timestamps or IDs might be the only change) |
| Convert to metric | Replace repeated logs with a metric to track counts or trends over time. | Logs that always show the same message or status. |
Review and apply recommendations
Navigate to the Log Optimizer page to view log patterns, sample messages, volume data, and plain-language explanations for each recommendation.
To apply a recommendation:
- Click a recommendation to open the side panel.
- Click an action button (Exclude Logs, Sample Logs, or Create Metric).
The change takes effect immediately in your configuration. However, the Log Optimizer page does not refresh until the next daily analysis runs, so the recommendation may still appear temporarily.
Additionally, create a ticket to start a review with other teams in your organization. Open a Jira ticket or create a Case with Datadog Case Management. For recommendations that you have addressed, mark them as resolved to hide from the recommendations feed.
When you review the Log Optimizer page, you notice a high-volume pattern from the shopist-support service. The message “Verifying ticket” appears over 1.3 million times each day across multiple hosts.
Datadog detects this as a repetitive pattern that doesn’t change and recommends converting it to a metric and excluding the log from indexing. You review the recommendation, confirm that these logs are repetitive, and apply the exclusion directly from the Recommendation side panel.
Critical error logs from the same service remain visible, allowing you to focus on meaningful signals without losing observability. After the next daily analysis, your updated configuration would show a reduction in indexed volume.
Track applied changes
Applying a recommendation creates an exclusion filter or a metric from the log. To see that filter or metric definition, change it, or remove it, go to the corresponding page:
You can edit or remove these configurations at any time from their respective pages.
Further reading
Additional helpful documentation, links, and articles:
