# Datafold ## Docs - [Get Audit Logs](https://docs.datafold.com/api-reference/audit-logs/get-audit-logs.md): Retrieve audit logs for your Datafold organization via the API. - [Create a DBT BI integration](https://docs.datafold.com/api-reference/bi/create-a-dbt-bi-integration.md): Create a dbt BI integration for lineage tracking via the Datafold API. - [Create a Hightouch integration](https://docs.datafold.com/api-reference/bi/create-a-hightouch-integration.md): Create a Hightouch integration for lineage tracking via the Datafold API. - [Create a Looker integration](https://docs.datafold.com/api-reference/bi/create-a-looker-integration.md): Create a Looker BI integration for lineage tracking via the Datafold API. - [Create a Mode Analytics integration](https://docs.datafold.com/api-reference/bi/create-a-mode-analytics-integration.md): Create a Mode Analytics BI integration for lineage tracking via the Datafold API. - [Create a Power BI integration](https://docs.datafold.com/api-reference/bi/create-a-power-bi-integration.md) - [Create a Tableau integration](https://docs.datafold.com/api-reference/bi/create-a-tableau-integration.md): Create a Tableau BI integration for lineage tracking via the Datafold API. - [Get an integration](https://docs.datafold.com/api-reference/bi/get-an-integration.md): Retrieve details of a specific BI integration by ID via the Datafold API. - [List all integrations](https://docs.datafold.com/api-reference/bi/list-all-integrations.md): List all BI integrations configured in your Datafold organization via the API. - [Remove an integration](https://docs.datafold.com/api-reference/bi/remove-an-integration.md): Remove a BI integration by ID via the Datafold API. - [Sync a BI integration](https://docs.datafold.com/api-reference/bi/sync-a-bi-integration.md): Trigger a sync for a specific BI integration via the Datafold API. - [Update a DBT BI integration](https://docs.datafold.com/api-reference/bi/update-a-dbt-bi-integration.md): Update an existing dbt BI integration via the Datafold API. - [Update a Hightouch integration](https://docs.datafold.com/api-reference/bi/update-a-hightouch-integration.md): Update an existing Hightouch integration via the Datafold API. - [Update a Looker integration](https://docs.datafold.com/api-reference/bi/update-a-looker-integration.md): Update an existing Looker BI integration via the Datafold API. - [Update a Mode Analytics integration](https://docs.datafold.com/api-reference/bi/update-a-mode-analytics-integration.md): Update an existing Mode Analytics BI integration via the Datafold API. - [Update a Power BI integration](https://docs.datafold.com/api-reference/bi/update-a-power-bi-integration.md): Updates the integration configuration. Returns the integration with changed fields. - [Update a Tableau integration](https://docs.datafold.com/api-reference/bi/update-a-tableau-integration.md): Update an existing Tableau BI integration via the Datafold API. - [List CI runs](https://docs.datafold.com/api-reference/ci/list-ci-runs.md): List all CI runs for a given CI configuration via the Datafold API. - [Trigger a PR/MR run](https://docs.datafold.com/api-reference/ci/trigger-a-prmr-run.md): Trigger a PR/MR diff run for a CI configuration via the Datafold API. - [Upload PR/MR changes](https://docs.datafold.com/api-reference/ci/upload-prmr-changes.md): Upload PR/MR changes for a specific pull request via the Datafold API. - [Cancel a running data diff](https://docs.datafold.com/api-reference/data-diffs/cancel-a-running-data-diff.md): Cancels a data diff that is currently queued or running. - [Create a data diff](https://docs.datafold.com/api-reference/data-diffs/create-a-data-diff.md): Create a new data diff to compare datasets via the Datafold API. - [Get a data diff](https://docs.datafold.com/api-reference/data-diffs/get-a-data-diff.md): Retrieve details of a specific data diff by ID via the Datafold API. - [Get a data diff summary](https://docs.datafold.com/api-reference/data-diffs/get-a-data-diff-summary.md): Get the summary results of a specific data diff via the Datafold API. - [Get a human-readable summary of a DataDiff comparison](https://docs.datafold.com/api-reference/data-diffs/get-a-human-readable-summary-of-a-datadiff-comparison.md): Retrieves a comprehensive, human-readable summary of a completed data diff. - [List data diffs](https://docs.datafold.com/api-reference/data-diffs/list-data-diffs.md): List all data diffs in your Datafold organization via the API. - [Update a data diff](https://docs.datafold.com/api-reference/data-diffs/update-a-data-diff.md): Update the configuration of an existing data diff via the Datafold API. - [Create a data source](https://docs.datafold.com/api-reference/data-sources/create-a-data-source.md): Create a new data source connection via the Datafold API. - [Execute a SQL query against a data source](https://docs.datafold.com/api-reference/data-sources/execute-a-sql-query-against-a-data-source.md): Executes a SQL query against the specified data source and returns the results. - [Get a data source](https://docs.datafold.com/api-reference/data-sources/get-a-data-source.md): Retrieve details of a specific data source by ID via the Datafold API. - [Get a data source summary](https://docs.datafold.com/api-reference/data-sources/get-a-data-source-summary.md): Get a summary of a specific data source by ID via the Datafold API. - [Get data source testing results](https://docs.datafold.com/api-reference/data-sources/get-data-source-testing-results.md): Retrieve the testing results for a data source connection via the Datafold API. - [List data source types](https://docs.datafold.com/api-reference/data-sources/list-data-source-types.md): List all supported data source types available in the Datafold API. - [List data sources](https://docs.datafold.com/api-reference/data-sources/list-data-sources.md): List all configured data sources in your Datafold organization via the API. - [Test a data source connection](https://docs.datafold.com/api-reference/data-sources/test-a-data-source-connection.md): Test the connection of a specific data source via the Datafold API. - [Datafold API](https://docs.datafold.com/api-reference/datafold-api.md): Datafold REST API reference for programmatic access to data diffs, data sources, CI runs, monitors, BI integrations, and more. - [Datafold SDK](https://docs.datafold.com/api-reference/datafold-sdk.md): Use the Datafold SDK for programmatic access to data diffs, CI artifact uploads, and integration with your data pipelines. - [Get column downstreams](https://docs.datafold.com/api-reference/explore/get-column-downstreams.md): Retrieve a list of columns or tables which depend on the given column. - [Get column upstreams](https://docs.datafold.com/api-reference/explore/get-column-upstreams.md): Retrieve a list of columns or tables which the given column depends on. - [Get table downstreams](https://docs.datafold.com/api-reference/explore/get-table-downstreams.md): Retrieve a list of tables which depend on the given table. - [Get table upstreams](https://docs.datafold.com/api-reference/explore/get-table-upstreams.md): Retrieve a list of tables which the given table depends on. - [Introduction](https://docs.datafold.com/api-reference/introduction.md): Get started with the Datafold REST API. Learn how to authenticate, obtain an API key, and make your first API call. - [MCP Server](https://docs.datafold.com/api-reference/mcp-server-setup.md): Connect AI assistants to Datafold using the Model Context Protocol - [Create a Data Diff Monitor](https://docs.datafold.com/api-reference/monitors/create-a-data-diff-monitor.md) - [Create a Data Test Monitor](https://docs.datafold.com/api-reference/monitors/create-a-data-test-monitor.md) - [Create a Metric Monitor](https://docs.datafold.com/api-reference/monitors/create-a-metric-monitor.md) - [Create a Schema Change Monitor](https://docs.datafold.com/api-reference/monitors/create-a-schema-change-monitor.md) - [Delete a Monitor](https://docs.datafold.com/api-reference/monitors/delete-a-monitor.md) - [Get Monitor](https://docs.datafold.com/api-reference/monitors/get-monitor.md) - [Get Monitor Run](https://docs.datafold.com/api-reference/monitors/get-monitor-run.md) - [List Monitor Runs](https://docs.datafold.com/api-reference/monitors/list-monitor-runs.md) - [List Monitors](https://docs.datafold.com/api-reference/monitors/list-monitors.md) - [Toggle a Monitor](https://docs.datafold.com/api-reference/monitors/toggle-a-monitor.md) - [Trigger a run](https://docs.datafold.com/api-reference/monitors/trigger-a-run.md) - [Update a Monitor](https://docs.datafold.com/api-reference/monitors/update-a-monitor.md) - [Best Practices](https://docs.datafold.com/data-diff/cross-database-diffing/best-practices.md): When dealing with large datasets, it's crucial to approach diffing with specific optimization strategies in mind. We share best practices that will help you get the most accurate and efficient results from your data diffs. - [Creating a New Data Diff](https://docs.datafold.com/data-diff/cross-database-diffing/creating-a-new-data-diff.md): Datafold's Data Diff can compare data across databases (e.g., PostgreSQL <> Snowflake, or between two SQL Server instances) to validate migrations, meet regulatory and compliance requirements, or ensure data is flowing successfully from source to target. - [Results](https://docs.datafold.com/data-diff/cross-database-diffing/results.md): Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab. - [How Datafold Diffs Data](https://docs.datafold.com/data-diff/how-datafold-diffs-data.md): Data diffs allow you to perform value-level comparisons between any two datasets within the same database, across different databases, or even between files. - [Best Practices](https://docs.datafold.com/data-diff/in-database-diffing/best-practices.md): We share best practices that will help you get the most accurate and efficient results from your data diffs. - [Creating a New Data Diff](https://docs.datafold.com/data-diff/in-database-diffing/creating-a-new-data-diff.md): Setting up a new data diff in Datafold is straightforward. - [Results](https://docs.datafold.com/data-diff/in-database-diffing/results.md): Once your data diff is complete, Datafold provides a concise, high-level summary of the detected changes in the Overview tab - [What's a Data Diff?](https://docs.datafold.com/data-diff/what-is-data-diff.md): A data diff is the value-level comparison between two tables, used to identify critical changes to your data and guarantee data quality. - [dbt Metadata Sync](https://docs.datafold.com/data-explorer/best-practices/dbt-metadata-sync.md): Datafold can automatically ingest dbt metadata from your production environment and display it in Data Explorer. - [How It Works](https://docs.datafold.com/data-explorer/how-it-works.md): Datafold's Data Knowledge Graph maps your entire data ecosystem — lineage, business logic, usage, and ontology — providing essential context to your AI agents via MCP and helping you understand the impact of changes across systems. - [Lineage](https://docs.datafold.com/data-explorer/lineage.md): Datafold offers a column-level and tabular lineage view. - [Profile](https://docs.datafold.com/data-explorer/profile.md): View a data profile that summarizes key table and column-level statistics, and any upstream dependencies. - [Datafold Migration Agent](https://docs.datafold.com/data-migration-automation/datafold-migration-agent.md): The Data Migration Agent delivers guaranteed-outcome migrations with fixed price, timeline, and data parity — over 6x faster than traditional approaches. - [Migration Automation](https://docs.datafold.com/data-migration-automation/datafold-migration-automation.md): Modernize your data platform in weeks, not years. Datafold's Data Migration Agent delivers guaranteed-outcome migrations with fixed price, timeline, and data parity — over 6x faster and cheaper than traditional approaches. - [Monitor Types](https://docs.datafold.com/data-monitoring/monitor-types.md): Monitoring your data for unexpected changes is one of the cornerstones of data observability. - [Monitors as Code](https://docs.datafold.com/data-monitoring/monitors-as-code.md): Manage Datafold monitors via version-controlled YAML for greater scalability, governance, and flexibility in code-based workflows. - [Data Diff Monitors](https://docs.datafold.com/data-monitoring/monitors/data-diff-monitors.md): Data Diff monitors compare datasets across or within databases, identifying row and column discrepancies with customizable scheduling and notifications. - [Data Test Monitors](https://docs.datafold.com/data-monitoring/monitors/data-test-monitors.md): Data Tests validate your data against off-the-shelf checks or custom business rules. - [Metric Monitors](https://docs.datafold.com/data-monitoring/monitors/metric-monitors.md): Metric monitors detect anomalies in your data using ML-based algorithms or manual thresholds, supporting standard and custom metrics for tables or columns. - [Schema Change Monitors](https://docs.datafold.com/data-monitoring/monitors/schema-change-monitors.md): Schema Change monitors notify you when a table’s schema changes, such as when columns are added, removed, or data types are modified. - [Deployment Options](https://docs.datafold.com/datafold-deployment/datafold-deployment-options.md): Datafold is a web-based application with multiple deployment options, including multi-tenant SaaS and dedicated cloud (either customer- or Datafold-hosted). - [Datafold VPC Deployment on AWS](https://docs.datafold.com/datafold-deployment/dedicated-cloud/aws.md): Learn how to deploy Datafold in a Virtual Private Cloud (VPC) on AWS. - [Datafold VPC Deployment on Azure](https://docs.datafold.com/datafold-deployment/dedicated-cloud/azure.md): Learn how to deploy Datafold in a Virtual Private Cloud (VPC) on Azure. - [Datafold VPC Deployment on GCP](https://docs.datafold.com/datafold-deployment/dedicated-cloud/gcp.md): Learn how to deploy Datafold in a Virtual Private Cloud (VPC) on GCP. - [MCP](https://docs.datafold.com/datafold-mcp.md): Connect your AI agent to Datafold and interact with your data through natural language - [AI Code Reviews](https://docs.datafold.com/deployment-testing/ai-code-reviews.md): Get automated, AI-powered code reviews on every pull request to catch SQL and data pipeline issues before they reach production. - [Handling Data Drift](https://docs.datafold.com/deployment-testing/best-practices/handling-data-drift.md): Ensuring Datafold in CI executes apples-to-apples comparison between staging and production environments. - [Slim Diff](https://docs.datafold.com/deployment-testing/best-practices/slim-diff.md): Choose which downstream tables to diff to optimize time, cost, and performance. - [Configuration](https://docs.datafold.com/deployment-testing/configuration.md): Explore configuration options for CI/CD testing in Datafold. - [Column Remapping](https://docs.datafold.com/deployment-testing/configuration/column-remapping.md): Specify column renaming in your git commit message so Datafold can map renamed columns to their original counterparts in production for accurate comparison. - [Running Data Diff for Specific PRs/MRs](https://docs.datafold.com/deployment-testing/configuration/datafold-ci/on-demand.md): By default, Datafold CI runs on every new pull/merge request and commits to existing ones. - [Running Data Diff on Specific Branches](https://docs.datafold.com/deployment-testing/configuration/datafold-ci/specifc.md): By default, Datafold CI runs on every new pull/merge request and commits to existing ones. - [Diff Timeline](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/diff-timeline.md): Specify a `time_column` to visualize match rates between tables for each column over time. - [Excluding Models](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/excluding-models.md): Use `never_diff` to exclude a model or subdirectory of models from data diffs. - [Including/Excluding Columns](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/including-excluding-columns.md): Specify columns to include or exclude from the data diff using `include_columns` and `exclude_columns`. - [SQL Filters](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/sql-filters.md): Use dbt YAML configuration to set model-specific filters for Datafold CI. - [Time Travel](https://docs.datafold.com/deployment-testing/configuration/model-specific-ci/time-travel.md): Use `prod_time_travel` and `pr_time_travel` to diff tables from specific points in time. - [Primary Key Inference](https://docs.datafold.com/deployment-testing/configuration/primary-key.md): Datafold requires a primary key to perform data diffs. Using dbt metadata, Datafold identifies the column to use as the primary key for accurate data diffs. - [Getting Started with CI/CD Testing](https://docs.datafold.com/deployment-testing/getting-started.md): Learn how to set up CI/CD testing with Datafold by integrating your data connections, code repositories, and CI pipeline for automated testing. - [API](https://docs.datafold.com/deployment-testing/getting-started/universal/api.md): Learn how to set up and configure Datafold's API for CI/CD testing. - [No-Code](https://docs.datafold.com/deployment-testing/getting-started/universal/no-code.md): Set up Datafold's No-Code CI integration to create and manage Data Diffs without writing code. - [How Datafold in CI Works](https://docs.datafold.com/deployment-testing/how-it-works.md): Learn how Datafold integrates with your Continuous Integration (CI) process with Data Diffs and AI Code Reviews, catching issues before they make it into production. - [CI/CD Testing](https://docs.datafold.com/faq/ci-cd-testing.md): Frequently asked questions about Datafold's CI/CD testing integration, including staging environments, diff performance, and automated data quality checks. - [Data Diffing](https://docs.datafold.com/faq/data-diffing.md): Frequently asked questions about Datafold's data diffing capabilities, including supported databases, data types, performance, and use cases. - [Data Monitoring and Observability](https://docs.datafold.com/faq/data-monitoring-observability.md): Frequently asked questions about Datafold's data monitoring and observability capabilities, including how it compares to other data observability tools. - [Data Reconciliation](https://docs.datafold.com/faq/data-reconciliation.md): Frequently asked questions about cross-database data reconciliation with Datafold, including how diffing works, scaling, and handling schema differences. - [Data Storage and Security](https://docs.datafold.com/faq/data-storage-and-security.md) - [Integrating Datafold with dbt](https://docs.datafold.com/faq/datafold-with-dbt.md): Frequently asked questions about using Datafold with dbt, including CI/CD setup for dbt Core and dbt Cloud, data diff performance, and testing workflows. - [Overview](https://docs.datafold.com/faq/overview.md): Get answers to the most common questions regarding our product. - [Performance and Scalability](https://docs.datafold.com/faq/performance-and-scalability.md) - [Resource Management](https://docs.datafold.com/faq/resource-management.md): Frequently asked questions about Datafold's resource consumption, data warehouse cost impact, and performance optimization. - [Hightouch](https://docs.datafold.com/integrations/bi-data-apps/hightouch.md): Navigate to Settings > Integrations > Data Apps and add a Hightouch Integration. - [Looker](https://docs.datafold.com/integrations/bi-data-apps/looker.md): Integrate Datafold with Looker to track BI lineage and understand the downstream impact of data changes on your Looker dashboards and Explores. - [Mode](https://docs.datafold.com/integrations/bi-data-apps/mode.md) - [Power BI](https://docs.datafold.com/integrations/bi-data-apps/power-bi.md): Include Power BI entities in Data Explorer and column-level lineage. - [Tableau](https://docs.datafold.com/integrations/bi-data-apps/tableau.md): Visualize downstream Tableau dependencies and understand how warehouse changes impact your BI layer. - [Tracking Jobs](https://docs.datafold.com/integrations/bi-data-apps/tracking-jobs.md): Track the completion and success of your data app integration syncs. - [Integrate with Code Repositories](https://docs.datafold.com/integrations/code-repositories.md): Connect your code repositories with Datafold. - [Azure DevOps](https://docs.datafold.com/integrations/code-repositories/azure-devops.md) - [Bitbucket](https://docs.datafold.com/integrations/code-repositories/bitbucket.md) - [GitHub](https://docs.datafold.com/integrations/code-repositories/github.md): Connect Datafold to GitHub to enable automated data diffs on pull requests, CI/CD testing integration, and code-level lineage tracking. - [GitLab](https://docs.datafold.com/integrations/code-repositories/gitlab.md) - [Set Up Your Data Connection](https://docs.datafold.com/integrations/databases.md): Set up your Data Connection with Datafold. - [Athena](https://docs.datafold.com/integrations/databases/athena.md) - [BigQuery](https://docs.datafold.com/integrations/databases/bigquery.md) - [Databricks](https://docs.datafold.com/integrations/databases/databricks.md): Connect Datafold to Databricks for data diffing, CI/CD testing, lineage, and migration validation. Includes setup instructions and required permissions. - [Dremio](https://docs.datafold.com/integrations/databases/dremio.md) - [MySQL](https://docs.datafold.com/integrations/databases/mysql.md) - [Netezza](https://docs.datafold.com/integrations/databases/netezza.md) - [Oracle](https://docs.datafold.com/integrations/databases/oracle.md): Connect Datafold to Oracle Database for data diffing, reconciliation, and migration validation. Includes setup instructions and required permissions. - [PostgreSQL](https://docs.datafold.com/integrations/databases/postgresql.md) - [Redshift](https://docs.datafold.com/integrations/databases/redshift.md) - [SAP HANA](https://docs.datafold.com/integrations/databases/sap-hana.md) - [Snowflake](https://docs.datafold.com/integrations/databases/snowflake.md): Connect Datafold to Snowflake for data diffing, CI/CD testing, lineage, and migration validation. Includes setup instructions and required permissions. - [Microsoft SQL Server](https://docs.datafold.com/integrations/databases/sql-server.md): Connect Datafold to Microsoft SQL Server for data diffing, reconciliation, and migration validation. Includes setup instructions and required permissions. - [Starburst](https://docs.datafold.com/integrations/databases/starburst.md) - [Teradata](https://docs.datafold.com/integrations/databases/teradata.md) - [OAuth Support](https://docs.datafold.com/integrations/oauth.md): Set up OAuth App Connections in your supported data warehouses to securely execute data diffs on behalf of your users. - [Integrate with Orchestrators](https://docs.datafold.com/integrations/orchestrators.md): Integrate Datafold with dbt Core, dbt Cloud, Airflow, or custom orchestrators to streamline your data workflows with automated monitoring, testing, and seamless CI integration. - [Custom Integrations](https://docs.datafold.com/integrations/orchestrators/custom-integrations.md): Integrate Datafold with your custom orchestration using the Datafold SDK and REST API. - [dbt Cloud](https://docs.datafold.com/integrations/orchestrators/dbt-cloud.md): Integrate Datafold with dbt Cloud to automate Data Diffs in your CI pipeline, leveraging dbt jobs to detect changes and ensure data quality before merging. - [dbt Core](https://docs.datafold.com/integrations/orchestrators/dbt-core.md): Set up Datafold’s integration with dbt Core to automate Data Diffs in your CI pipeline. - [Compliance & Trust Center](https://docs.datafold.com/security/compilance-trust-center.md) - [MCP Tool Permissions](https://docs.datafold.com/security/mcp-tool-permissions.md): Which permissions each MCP tool requires. Use this reference when scoping a service account's group for MCP use. - [Securing Connections](https://docs.datafold.com/security/securing-connections.md): Datafold supports multiple options to secure connections between your resources (e.g., databases and BI tools) and Datafold. - [Service Accounts](https://docs.datafold.com/security/service-accounts.md): Machine identities for CI, integrations, and scripts. Service accounts own their own API keys, inherit permissions from groups, and are managed independently of human users. - [Single Sign-On](https://docs.datafold.com/security/single-sign-on.md): Set up Single Sign-On with one of the following options. - [Google OAuth](https://docs.datafold.com/security/single-sign-on/google-oauth.md): Configure Google OAuth single sign-on (SSO) for Datafold. Step-by-step setup instructions for authenticating your team with Google. - [Okta (OIDC)](https://docs.datafold.com/security/single-sign-on/okta.md): Configure Okta OIDC single sign-on (SSO) for Datafold. Step-by-step setup instructions for authenticating your team with Okta. - [SAML](https://docs.datafold.com/security/single-sign-on/saml.md): SAML (Security Assertion Markup Language) is a protocol that enables secure user authentication by integrating Identity Providers (IdPs) with Service Providers (SPs). - [Google](https://docs.datafold.com/security/single-sign-on/saml/examples/google.md) - [Microsoft Entra ID](https://docs.datafold.com/security/single-sign-on/saml/examples/microsoft-entra-id-configuration.md): Configure Microsoft Entra ID (Azure AD) as a SAML identity provider for Datafold SSO. Step-by-step setup and configuration guide. - [Okta](https://docs.datafold.com/security/single-sign-on/saml/examples/okta.md) - [Group provisioning](https://docs.datafold.com/security/single-sign-on/saml/group-provisioning.md): Automatically sync group membership with your SAML Identity Provider (IdP). - [User Roles and Permissions](https://docs.datafold.com/security/user-roles-and-permissions.md): Datafold uses role-based access control to manage user permissions and actions. - [FAQ](https://docs.datafold.com/support/faq-redirect.md) - [Support](https://docs.datafold.com/support/support.md): Datafold offers multiple support channels to assist users with troubleshooting and inquiries. - [Datafold](https://docs.datafold.com/welcome.md): Datafold is the data engineering automation platform that combines specialized AI agents with a context layer and data quality tools — so data teams and their coding agents ship higher-quality data faster, migrate with confidence, and optimize platform costs. ## OpenAPI Specs - [openapi-public](https://docs.datafold.com/openapi-public.json) - [openapi](https://docs.datafold.com/api-reference/openapi.json) ## Optional - [About Datafold](https://www.datafold.com/) - [Blog](https://www.datafold.com/blog?)