Apache Airflow version

3.1.6

If "Other Airflow 3 version" selected, which one?

No response

What happened?

This is a regression from Airflow 2. In Airflow 2, the UI properly extracted and displayed SQL queries from BigQuery job configurations as separate, formatted fields. In Airflow 3, this functionality has been lost.

When using BigQueryInsertJobOperator, the Rendered Templates view only displays the full configuration field as JSON, without extracting the SQL query into a separate field.

The SQL query is embedded within the JSON configuration as an escaped string (e.g., "query": "\\nSELECT\\n customer_id,\\n..."), making it impossible to:

• Quickly copy the SQL query for debugging or testing
• View the SQL with proper formatting and syntax highlighting
• Avoid dealing with JSON escape characters

This is a regression from the expected behaviour where nested template fields defined in template_fields_renderers should be displayed as separate entries.

What you think should happen instead?

The Rendered Templates view should display two separate entries:

1. configuration - The full job configuration as formatted JSON
2. configuration.query.query - The extracted SQL query with SQL syntax highlighting

This would match the operator's template_fields_renderers configuration:

template_fields_renderers = {
    "configuration": "json",
    "configuration.query.query": "sql"  # Should create a separate SQL entry
}

Users should be able to:

• See the SQL query as a standalone field with proper SQL formatting
• Copy the entire SQL with one click using the copy button
• Avoid manually extracting SQL from nested JSON strings

How to reproduce

1. Create a DAG with BigQueryInsertJobOperator containing a multi-line SQL query:

from datetime import datetime
from airflow import DAG
from airflow.providers.google.cloud.operators.bigquery import BigQueryInsertJobOperator

COMPLEX_SQL = """
SELECT
    customer_id,
    customer_name,
    order_date,
    SUM(order_amount) as total_amount,
    COUNT(DISTINCT order_id) as order_count
FROM
    `project.dataset.orders` o
    INNER JOIN `project.dataset.customers` c
        ON o.customer_id = c.id
WHERE
    order_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
    AND order_status = 'completed'
GROUP BY
    customer_id,
    customer_name,
    order_date
ORDER BY
    total_amount DESC
LIMIT 100
"""

with DAG(
    dag_id="bigquery_sql_rendering_test",
    start_date=datetime(2024, 1, 1),
    schedule=None,
    catchup=False,
) as dag:
    run_query = BigQueryInsertJobOperator(
        task_id="run_query",
        configuration={
            "query": {
                "query": COMPLEX_SQL,
                "useLegacySql": False,
            }
        },
        project_id="my-project",
        location="US",
    )

2. Trigger the DAG manually from the Airflow UI
3. Wait for the task to complete (it will likely fail due to invalid credentials, but that's okay)
4. Navigate to: Grid View → Click on the task instance → "Rendered Templates" tab
5. Observe: Only the configuration field is shown as JSON with the SQL embedded inside

Expected: Should see both configuration (JSON) and configuration.query.query (SQL) as separate entries

Actual: Only see configuration (JSON) with SQL embedded as an escaped string within

Operating System

Debian GNU/Linux 12 (bookworm)

Versions of Apache Airflow Providers

No response

Deployment

Other

Deployment details

No response

Anything else?

This issue is related to:

• #39527 - Restored the Rendered Templates view in Grid
• #49064 - Added template_fields_renderers support in UI

While both issues were resolved, the specific problem of extracting nested template fields (using dot notation like configuration.query.query) remains unaddressed.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct