Getting Started with Database Monitoring
Overview
Datadog Database Monitoring helps you to better understand the health and performance of your databases and to determine the root cause of any problems.
In one place, you can view:
- Host-level metrics
- Explain plans
- Historical query performance metrics
Work through this guide to set up Datadog Database Monitoring on an example PostgreSQL database. Next, identify an expensive query, troubleshoot a slow query, and create a dashboard to view changes in query volume.
Setup
Prerequisites
Before getting started, you need a Datadog account.
To run the example application, you need a machine with GNU Make and Docker Desktop. Have your Datadog API key available.
Install the example application
The example application runs three containers: a PostgreSQL or MySQL database, the Datadog Agent, and a Go orders app that continuously generates realistic database traffic. While the app runs, the Agent sends the following signals to Datadog:
| Signal | Description |
|---|
| Query metrics | Per-query latency, execution count, and rows examined. |
| Explain plans | Execution plans sampled at runtime, surfacing costly operations. |
| Full table scan | The SELECT * FROM orders WHERE status = ? query has no index on status, causing a sequential scan. |
| Lock contention | The app uses SELECT ... FOR UPDATE to simulate concurrent row locks. |
| APM ↔ DBM correlation | Traces from the orders service are linked to their database queries, enabling end-to-end visibility. |
You can view this data in Datadog Database Monitoring.
Follow these instructions to install the example application on macOS or Linux.
Clone the repository containing the example application:
git clone https://github.com/DataDog/dd-database-monitoring-example
Change to the dd-database-monitoring-example directory:
cd dd-database-monitoring-example
Set the environment variable DD_API_KEY to your Datadog API key:
export DD_API_KEY=<API_KEY>
Start the application. Choose the database you want to monitor:
If your Datadog account is not on the US1 site, also export DD_SITE before running make:
export DD_SITE=datadoghq.eu # example: EU site
The command continues to run until you stop it by pressing Ctrl + C.
Identify an expensive query
Which query consumes the most database time? To find out, use the Query Metrics view.
On the Database Monitoring page, click the Query metrics tab in the UI.
Sort the Normalized Query table by Percent time to see the query that the database spends the most time executing.
The query that consumes the most database time appears on the first line. Look for the query scanning the orders table by status—for example, SELECT * FROM orders WHERE status = ?. This query is expensive because there is no index on the status column, so the database performs a full sequential scan of the table on every execution.
Troubleshoot a slow query
In addition to identifying slow queries, Datadog Database Monitoring can help you diagnose them. A query’s Explain Plan describes the steps that the database takes to resolve the query. View an Explain Plan by clicking on a sample in the Query Samples view.
Navigate to the Query Samples view within Database Monitoring by selecting the Samples tab.
In the In dropdown, select Explain Plans.
Sort the Normalized Query table by Duration.
Find a query in the table with data in the Explain Plan column and click on it to open the Sample Details page.
Under Explain Plan, click List View. The list view breaks down each step the database takes to execute the query—the operation type, the table it runs against, and cost estimates—giving you a starting point to understand why a query is slow.
Correlate traces and database queries
The orders app emits APM traces that are automatically linked to the database queries they generate. Use this correlation to move seamlessly between a slow trace and the exact query responsible.
Navigate to APM > Traces and find a trace from the orders service.
On the trace flame graph, click a database span to open its details panel.
In the details panel, click View in DBM to jump directly to that query in Database Monitoring.
Conversely, in a Query Sample in Database Monitoring, click View Trace to open the originating APM trace.
To understand the health and performance of your databases at a glance, add Datadog Database Monitoring metrics to a dashboard.
View changes in query volume
For example, you can see the absolute change in query volume in the past hour by adding a Change widget to track a query count metric.
Select Dashboards > New Dashboard in the UI.
Enter a name for your dashboard. Click the New Dashboard button to go to your new dashboard.
To add content to your dashboard, click Add Widgets.
In the widget carousel, select the Change widget.
Select postgresql.queries.count in the Metric dropdown. This metric counts the number of queries sent to a PostgreSQL database. If you are monitoring MySQL, use mysql.queries instead.
Select host in the Break it down by dropdown so that the widget aggregates queries by host.
Click the Save button. The dashboard shows your new widget.
View out-of-the-box dashboards
Observe current database activity, resource utilization, and more on out-of-the-box dashboards provided by Datadog Database Monitoring.
To access the dashboards, from the Database Monitoring page, select the Dashboards tab and choose the dashboard that you want to see.
You can clone and modify out-of-the-box dashboards to suit your needs.
Further Reading
Additional helpful documentation, links, and articles:
